Symbolic links are added as symlinks to the sources JAR instead of being followed

[mirabilos]

I’ve got symbolic links in my resources (files that get reused in multiple places in a larger repository). They are correctly followed when creating the main binary JAR but the sources JAR contains the symbolic links themselves¹, which then point to nirvana (i.e. to outside the JAR).

In #47 (comment) it was suggested to make the “follow symlinks?” mode configurable. Implementing this would completely solve my issue if it is configurable in the necessary places; I believe that implementing it in the manner shown will fix it.

Related: https://issues.apache.org/jira/browse/MRESOURCES-237 where 3.x versions of the maven-resources-plugin cause similar problems by not following symlinks any more (which is why I’m pinning it to a 2.x version in my projects).

Fixing this in plexus-archiver might fix the maven-resources-plugin problem as well… unsure.

① Funnily enough, this even causes other software to crash when exploring the resulting archives…

[michael-o]

The reported crash isn't reproducible for me with:

$ mc -V
GNU Midnight Commander 4.8.24
Kompiliert mit GLib 2.66.2
Die ncurses-Bibliothek benutzen
Mit eingebautem Editor
Mit Unterstützung für Hintergrundtätigkeiten
Mit Maus-Unterstützung für xterm
Mit Unterstützung für X11-Ereignisse
Mit Internationalisierungs-Unterstützung
Mit Unterstützung mehrerer Codepages
Virtuelles Dateisystem: cpiofs, tarfs, sfs, extfs, ftpfs, fish
Datentyp: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

This needs to be verified with Plexus Archiver.

[mirabilos]

It’s not about the crash, this is just a funny side effect.

I need to be able to have the maven-source-plugin follow symlinks when creating the sources JAR instead of adding the symlinks themselves into the JAR. This is what this is about.

[michael-o]

Likely both modes would be required. Retention and resolution.

[mirabilos]

Yes, of course, hence the request to make it configurable.

Per resource set, as @petr-ujezdsky suggested in his comment, would probably match best.

[mirabilos]

Setting collection.setFollowingSymLinks(true); does not seem to be enough; the resulting JAR file still contains the symbolic link instead of the pointed-to file :(

[mirabilos]

Some debugging later…

Archiver archiver;
archiver.addDirectory( sourceDirectory, pIncludes, pExcludes );

        getLog().info( "resource {{{" );
        ResourceIterator ri = archiver.getResources();
        while (ri.hasNext()) {
            getLog().info( "resource: " + fmt(ri.next()) );
        }
        getLog().info( "resource }}}" );
    }

    private static String fmt(final ArchiveEntry e) {
    	return String.format("%d(%o)<%s>", e.getType(),e.getMode(),e.getName());
    }

… it clearly shows that, despite…

--- a/src/main/java/org/codehaus/plexus/archiver/AbstractArchiver.java
+++ b/src/main/java/org/codehaus/plexus/archiver/AbstractArchiver.java
@@ -367,7 +367,7 @@ public void addFileSet( @Nonnull final FileSet fileSet )
         // The PlexusIoFileResourceCollection contains platform-specific File.separatorChar which
         // is an interesting cause of grief, see PLXCOMP-192
         final PlexusIoFileResourceCollection collection = new PlexusIoFileResourceCollection();
-        collection.setFollowingSymLinks( false );
+        collection.setFollowingSymLinks( true );
 
         collection.setIncludes( fileSet.getIncludes() );
         collection.setExcludes( fileSet.getExcludes() );

…, the collection entry is still a symlink:

[INFO] indeed running the patched plugin
[INFO] resource {{{
[INFO] resource: 2(40755)<>
[INFO] resource: 2(40755)<org>
[INFO] resource: 2(40755)<org/evolvis>
[INFO] resource: 2(40755)<org/evolvis/tartools>
[INFO] resource: 2(40755)<org/evolvis/tartools/mvnparent>
[INFO] resource: 2(40755)<org/evolvis/tartools/mvnparent/examples>
[INFO] resource: 1(100644)<org/evolvis/tartools/mvnparent/InitialiseLogging.java>
[INFO] resource: 1(100644)<org/evolvis/tartools/mvnparent/examples/Main.java>
[INFO] resource: 2(40755)<META-INF>
[INFO] resource: 2(40755)<META-INF/legal>
[INFO] resource: 2(40755)<META-INF/legal/org.evolvis.tartools>
[INFO] resource: 2(40755)<META-INF/legal/org.evolvis.tartools/maven-parent-lib>
[INFO] resource: 3(120777)<META-INF/legal/org.evolvis.tartools/maven-parent-lib/LICENCE>
[INFO] resource }}}

This is about the last file, a symbolic link:

lrwxrwxrwx 1 me me 54 Jan  8 15:35 lib/src/main/resources/META-INF/legal/org.evolvis.tartools/maven-parent-lib/LICENCE -> ../../../../../../../../src/main/ancillary/LICENCE.hdr

I hope the fact that the amount of ../s brings it out of the lib/ subdirectory is not the problem here, because this is how files are shared in multi-module projects…

[mirabilos]

This might be due to plexus-io ResourceFactory…

public static PlexusIoResource createResource( File f, String name, final ContentSupplier contentSupplier,
                                               InputStreamTransformer inputStreamTransformer,
                                               PlexusIoResourceAttributes attributes )
    throws IOException
{
    boolean symbolicLink = attributes.isSymbolicLink();
    return symbolicLink ? new PlexusIoSymlinkResource( f, name, attributes )
        :  new PlexusIoFileResource(f, name, attributes, contentSupplier, inputStreamTransformer);
}

… in PlexusIoFileResourceCollection…

        File f = new File( dir, sourceDir );

        PlexusIoResourceAttributes attrs = new FileAttributes( f, cache1, cache2 );
        attrs = mergeAttributes( attrs, f.isDirectory() );

        String remappedName = getName( name );

        PlexusIoResource resource =
            ResourceFactory.createResource( f, remappedName, null, getStreamTransformer(), attrs );

… and FileAttributes…

        Map<String, Object> attrs = Files.readAttributes( path, "unix:permissions,gid,uid,isSymbolicLink,mode", LinkOption.NOFOLLOW_LINKS );

… so the collection.setFollowingSymLinks(true); seems to be only about following symbolic links to directories when encountered; additional remapping of symbolic links in plexus-io is needed.

[mirabilos]

I’m currently trying to implement this inside plexus-archiver in a configurable way (support both retention and resolution of symbolic links, as you said), but I have absolutely no idea how to then tie this in with the maven-source-plugin because the Maven Archiver configuration does not expose any plexus-archiver configuration, if there is any.

@michael-o can you help me with that?

[mirabilos]

My WIP branch on top of the 4.2.1 release (since that’s what the maven-source-plugin uses) now has this implemented, even extending the ZipArchiverTest.testSymlinkFileSet() case so it tests both retaining and resolving symlinks at creation time.

Even if this necessarily must wait for a new plexus-io release with that PR applied, review (also whether I’m on the correct path here) welcome…

[michael-o]

You need to pass this through all layers. All plugins using Maven Archiver expose XML configuration for it.

[mirabilos]

Michael Osipov dixit:

You need to pass this through all layers.

I guessed so. Right now, it’s available in the Archiver interface of plexus-archiver as of the current WIP.

All plugins using Maven Archiver expose XML configuration for it.

“For it” being the Maven Archiver, not the plexus-archiver, right? I saw configuration/XML documentation for that. I’m a bit overwhelmed should needing to completely add support to configure plexus-archiver from the Maven Archiver need to be created first… bye, //mirabilos -- tarent solutions GmbH Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/ Tel: +49 228 54881-393 • Fax: +49 228 54881-235 HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941 Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg