solves #35: allow to also whitelist source paths

Author: royteeuwen

aem-classification-validator/README.md

@@ -14,9 +14,10 @@ The following options are supported apart from the default settings mentioned in
 
 Option | Mandatory | Description
 --- | --- | ---
-maps | yes | a comma-separated list of URLs specifying the source for a classification map. Each URL might use the protocols `file:`, for file-based classification maps, `http(s):` for classification maps in the internet or `tccl:` for classification maps being provided via the ThreadContextClassloader. The latter is especially useful with Maven as the TCCL during the execution of a goal of a Maven Plugin is the [Maven Plugin Classpath][4].
-whitelistedResourcePathPatterns | no | a comma-separated list of regular expressions matching an absolute resource path which should not be reported (no matter if its usage violates content classifications or not). The path is referring to the referenced/inherited/overlaid resource path (not the path containing the reference/supertype/overlay). 
-severitiesPerClassification | no | the severity per classification (this will overwrite the default severity which otherwise used for all classifications). The format is `<classification>=<severity>{,<classification>=<severity>}`, where `classification` is one of `INTERNAL`, `INTERNAL_DEPRECATED_ANNOTATION`, `INTERNAL_DEPRECATED`, `FINAL` or `ABSTRACT` and `severity` is one of `DEBUG`, `INFO`, `WARN` or `ERROR`.
+ maps                            | yes       | a comma-separated list of URLs specifying the source for a classification map. Each URL might use the protocols `file:`, for file-based classification maps, `http(s):` for classification maps in the internet or `tccl:` for classification maps being provided via the ThreadContextClassloader. The latter is especially useful with Maven as the TCCL during the execution of a goal of a Maven Plugin is the [Maven Plugin Classpath][4]. 
+ whitelistedResourcePathPatterns | no        | a comma-separated list of regular expressions matching an absolute resource path which should not be reported (no matter if its usage violates content classifications or not). The path is referring to the referenced/inherited/overlaid resource path (not the path containing the reference/supertype/overlay).
+ whitelistedSourcePathPatterns   | no        | a comma-separated list of regular expressions matching an absolute source path which should not be reported (no matter if its usage violates content classifications or not). Use this if you know there is an issue with classification for a specific component, but you don't want the problem to spread to other components.
+ severitiesPerClassification     | no        | the severity per classification (this will overwrite the default severity which otherwise used for all classifications). The format is `<classification>=<severity>{,<classification>=<severity>}`, where `classification` is one of `INTERNAL`, `INTERNAL_DEPRECATED_ANNOTATION`, `INTERNAL_DEPRECATED`, `FINAL` or `ABSTRACT` and `severity` is one of `DEBUG`, `INFO`, `WARN` or `ERROR`.
 
 All validation messages are emitted with the [`defaultSeverity`][2]
 
@@ -33,16 +34,16 @@ The file is a CSV serialization of the map where each line represents one item i
 <path>,<classification>(,<remark>)
 ```
 
-where `classification` is one of 
+where `classification` is one of
 
 1. `INTERNAL`
 2. `INTERNAL_DEPRECATED_ANNOTATION`, same restrictions as `INTERNAL` but due to being marked as deprecated via some annotation e.g. `cq:deprecated` property
 3. `INTERNAL_DEPRECATED`, same restrictions as `INTERNAL` but due to being marked as deprecated in some external sources like release notes
 4. `FINAL`
 5. `ABSTRACT`
-6. `PUBLIC` 
+6. `PUBLIC`
 
-(in order from most restricted to least restricted). 
+(in order from most restricted to least restricted).
 The explanation for those can be found in the [Adobe documentation][1].
 The CSV format is based on [RFC 4180][7]. In addition a comment starting with `#` on the first line is supposed to contain a label for the map (like the underlying AEM version). `path` is supposed to be an absolute JCR path of a specific node.
 
@@ -94,8 +95,8 @@ There are several reasons:
 
 1. You should detect violations as early as possible, preferably already in your CI pipeline. The later you detect those the more effort it is to fix.
 2. If you don't care about content classifications
-   1. there is a high chance that you cannot easily upgrade to a newer AEM version (AMS or on-premise)
-   2. it might break with every new [AEM as a Cloud Service][5] release
+    1. there is a high chance that you cannot easily upgrade to a newer AEM version (AMS or on-premise)
+    2. it might break with every new [AEM as a Cloud Service][5] release
 
 [1]: https://docs.adobe.com/content/help/en/experience-manager-65/deploying/upgrading/sustainable-upgrades.html#content-classifications
 [2]: https://jackrabbit.apache.org/filevault/validation.html

aem-classification-validator/pom.xml

@@ -92,6 +92,12 @@
             <version>2.2</version>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>org.mockito</groupId>
+            <artifactId>mockito-core</artifactId>
+            <version>5.20.0</version>
+            <scope>test</scope>
+        </dependency>
         <!-- only transitive dependencies of 'vault-validation' but must be declared due to https://issues.apache.org/jira/browse/JCRVLT-394 -->
         <dependency>
             <groupId>javax.jcr</groupId>
@@ -112,4 +118,4 @@
             <scope>test</scope>
         </dependency>
     </dependencies>
-</project>
+</project>

aem-classification-validator/src/main/java/biz/netcentric/filevault/validator/aem/classification/AemClassificationValidator.java

@@ -41,21 +41,25 @@
 import org.apache.sling.jcr.resource.api.JcrResourceConstants;
 import org.jetbrains.annotations.NotNull;
 import org.jetbrains.annotations.Nullable;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 public class AemClassificationValidator implements DocumentViewXmlValidator, GenericJcrDataValidator, NodePathValidator {
 
+    private static final Logger LOGGER = LoggerFactory.getLogger(AemClassificationValidator.class);
+
     /**
      * Example HTL code which should be matched by the RegEx:
      * <br>
      * <code>&lt;article data-sly-resource="${'path/to/resource' @ resourceType='&lt;some resource type&gt;'}"&gt;</code>
      * <br>
      * The first subgroup must contain the value of the resource type.
      * This pattern only works if the resource type is given as literal!
-     * 
+     *
      * @see <a href="https://helpx.adobe.com/experience-manager/htl/using/block-statements.html#resource">data-sly-resource</a>
      */
     static final Pattern HTL_INCLUDE_OVERWRITING_RESOURCE_TYPE = Pattern.compile("data-sly-resource\\s*=[^@]*.*?resourceType\\s*=\\s*(?:\"|\')([^'\"]*)(?:\"|\')");
-    
+
     /**
      * Example JSP code which should be matched by the RegEx:
      * <br>
@@ -64,7 +68,7 @@ public class AemClassificationValidator implements DocumentViewXmlValidator, Gen
      * <br>
      * The first subgroup must contain the value of the resource type.
      * This pattern only works if the resource type is given as literal!
-     * 
+     *
      * @see <a href="https://experienceleague.adobe.com/docs/experience-manager-65/developing/platform/taglib.html?lang=en">CQ/Sling Tag Library</a>
      */
     private static final Pattern JSP_INCLUDE_OVERWRITING_RESOURCE_TYPE = Pattern.compile("(?:<cq:|<sling:)include resourceType\\s*=\\s*(?:\"|\')([^'\"]*)(?:\"|\')");
@@ -84,18 +88,22 @@ public class AemClassificationValidator implements DocumentViewXmlValidator, Gen
 
     private final ContentClassificationMap classificationMap;
     private final Collection<String> whitelistedResourcePaths;
+    private final Collection<String> whitelistedSourcePaths;
     private final Collection<Pattern> whitelistedResourcePathPatterns;
+    private final Collection<Pattern> whitelistedSourcePathPatterns;
     private final Map<ContentClassification, ValidationMessageSeverity> severityPerClassification;
 
     private @NotNull ValidationMessageSeverity defaultSeverity;
     private final Collection<String> overlaidNodePaths;
 
-    public AemClassificationValidator(@NotNull ValidationMessageSeverity defaultSeverity, @NotNull ContentClassificationMap classificationMap, @NotNull Collection<String> whitelistedResourcePaths, @NotNull Map<ContentClassification, ValidationMessageSeverity> severityPerClassification) {
+    public AemClassificationValidator(@NotNull ValidationMessageSeverity defaultSeverity, @NotNull ContentClassificationMap classificationMap, @NotNull Collection<String> whitelistedResourcePaths, @NotNull Collection<String> whitelistedSourcePaths, @NotNull Map<ContentClassification, ValidationMessageSeverity> severityPerClassification) {
         super();
         this.defaultSeverity = defaultSeverity;
         this.classificationMap = classificationMap;
         this.whitelistedResourcePaths = whitelistedResourcePaths;
         this.whitelistedResourcePathPatterns = whitelistedResourcePaths.stream().map(Pattern::compile).collect(Collectors.toList());
+        this.whitelistedSourcePaths = whitelistedSourcePaths;
+        this.whitelistedSourcePathPatterns = whitelistedSourcePaths.stream().map(Pattern::compile).collect(Collectors.toList());
         this.severityPerClassification = severityPerClassification;
         this.overlaidNodePaths = new LinkedList<>();
     }
@@ -106,6 +114,11 @@ public Collection<ValidationMessage> done() {
 
     @Override
     public Collection<ValidationMessage> validate(@NotNull String path) {
+        if (isSourcePathWhitelisted(path, whitelistedSourcePathPatterns)) {
+            LOGGER.debug("Source path '{}' is explicitly whitelisted and therefore has no restrictions!", path);
+            return null;
+        }
+
         if (!overlaidNodePaths.contains(path)) {
             // check overlay usage in addition for non-docview files
             ValidationMessage message = validateClassification(path,  ContentUsage.OVERLAY, MESSAGE_SUBJECT_FILE);
@@ -118,6 +131,10 @@ public Collection<ValidationMessage> validate(@NotNull String path) {
 
     @Override
     public boolean shouldValidateJcrData(@NotNull Path filePath) {
+        if (isSourcePathWhitelisted(filePath.toString(), whitelistedSourcePathPatterns)) {
+            LOGGER.debug("Source path '{}' is explicitly whitelisted and therefore has no restrictions!", filePath);
+            return false;
+        }
         return (isHtlFile(filePath) || isJspFile(filePath));
     }
 
@@ -167,6 +184,11 @@ static String jcrExpandedFormNameToReadableFormat(String jcrExpandedFormName) {
 
     @Override
     public Collection<ValidationMessage> validate(@NotNull DocViewNode node, @NotNull String nodePath, @NotNull Path filePath, boolean isRoot) {
+        if (isSourcePathWhitelisted(nodePath, whitelistedSourcePathPatterns)) {
+            LOGGER.debug("Source path '{}' is explicitly whitelisted and therefore has no restrictions!", nodePath);
+            return null;
+        }
+
         Collection<ValidationMessage> messages = new LinkedList<>();
         String subject = String.format(MESSAGE_SUBJECT_NODE, jcrExpandedFormNameToReadableFormat(node.label));
 
@@ -190,7 +212,7 @@ public Collection<ValidationMessage> validate(@NotNull DocViewNode node, @NotNul
            messages.add(message);
            overlaidNodePaths.add(nodePath);
         }
-        
+
         // TODO: check usage of clientlib dependencies/embeds
         return messages;
     }
@@ -204,7 +226,7 @@ public Collection<ValidationMessage> validate(@NotNull DocViewNode node, @NotNul
             // add subject and usage to message
             return new ValidationMessage(defaultSeverity, "Resource path must not end with '/' but is '" + resourcePath + "'");
         }
-        
+
         if (usage == ContentUsage.OVERLAY) {
             if (!resourcePath.startsWith("/apps/")) {
                 return null; // this is not an overlay at all, therefore no violation
@@ -228,26 +250,34 @@ private static boolean isJspFile(Path file) {
         return JSP_PATH_MATCHER.matches(file);
     }
 
+    private static boolean isSourcePathWhitelisted(@NotNull String sourcePath, @Nullable Collection<Pattern> whitelistedSourcePathPatterns) {
+        if (whitelistedSourcePathPatterns == null) {
+            return false;
+        }
+        return whitelistedSourcePathPatterns.stream().anyMatch(r -> r.matcher(sourcePath).matches());
+    }
+
     static @NotNull String extendMessageWithRemark(@NotNull String message, String remark) {
         if (remark != null && !remark.isEmpty()) {
             return message + " Remark: " + remark;
         }
         return message;
     }
-    
+
     @NotNull ValidationMessageSeverity getSeverityForClassification(ContentClassification classification) {
         ValidationMessageSeverity severity = severityPerClassification.get(classification);
         return severity != null ? severity : defaultSeverity;
     }
 
-    
+
     @Override
     public int hashCode() {
         final int prime = 31;
         int result = 1;
         result = prime * result + ((classificationMap == null) ? 0 : classificationMap.hashCode());
         result = prime * result + ((defaultSeverity == null) ? 0 : defaultSeverity.hashCode());
         result = prime * result + ((whitelistedResourcePaths == null) ? 0 : whitelistedResourcePaths.hashCode());
+        result = prime * result + ((whitelistedSourcePaths == null) ? 0 : whitelistedSourcePaths.hashCode());
         result = prime * result + ((severityPerClassification == null) ? 0 : severityPerClassification.hashCode());
         return result;
     }
@@ -273,6 +303,11 @@ public boolean equals(Object obj) {
                 return false;
         } else if (!whitelistedResourcePaths.equals(other.whitelistedResourcePaths))
             return false;
+        if (whitelistedSourcePaths == null) {
+            if (other.whitelistedSourcePaths != null)
+                return false;
+        } else if (!whitelistedSourcePaths.equals(other.whitelistedSourcePaths))
+            return false;
         if (severityPerClassification == null) {
             if (other.severityPerClassification != null)
                 return false;
@@ -285,6 +320,7 @@ public boolean equals(Object obj) {
     public String toString() {
         return "AemClassificationValidator [" + (classificationMap != null ? "classificationMap=" + classificationMap + ", " : "")
                 + (whitelistedResourcePaths != null ? "resourceTypeWhitelist=" + whitelistedResourcePaths + ", " : "")
+                + (whitelistedSourcePaths != null ? "sourcePathWhitelist=" + whitelistedSourcePaths + ", " : "")
                 + (severityPerClassification != null ? "severityPerClassification=" + severityPerClassification + ", " : "")
                 + (defaultSeverity != null ? "defaultSeverity=" + defaultSeverity : "") + "]";
     }

aem-classification-validator/src/main/java/biz/netcentric/filevault/validator/aem/classification/AemClassificationValidatorFactory.java

@@ -52,6 +52,9 @@ public class AemClassificationValidatorFactory implements ValidatorFactory {
     private static final String OPTION_WHITELISTED_RESOURCE_PATH_PATTERNS = "whitelistedResourcePathPatterns";
     private static final String OPTION_WHITELISTED_RESOURCE_PATH_PATTERNS_OLD = "whitelistedResourcePathsPatterns";
 
+    /** optional list of comma-separated source path patterns (should be absolute) */
+    private static final String OPTION_WHITELISTED_SOURCE_PATH_PATTERNS = "whitelistedSourcePathPatterns";
+
     private static final Object OPTION_SEVERITIES_PER_CLASSIFICATION = "severitiesPerClassification";
 
     private static final Logger LOGGER = LoggerFactory.getLogger(AemClassificationValidatorFactory.class);
@@ -72,18 +75,14 @@ public Validator createValidator(@NotNull ValidationContext context, @NotNull Va
             optionWhitelistedResourcePaths = settings.getOptions().get(OPTION_WHITELISTED_RESOURCE_PATH_PATTERNS);
         }
 
-        Collection<String> whitelistedResourcePaths =
-                Optional.ofNullable(optionWhitelistedResourcePaths)
-                        .map(op -> Arrays.stream(op.split(","))
-                                .map(String::trim)
-                                .collect(Collectors.toList()))
-                        .orElse(Collections.emptyList());
-
-        try {
-            whitelistedResourcePaths.stream().forEach(AemClassificationValidatorFactory::validateResourcePathPattern);
-        } catch (IllegalArgumentException e) {
-            throw new IllegalArgumentException("At least one value given in option " + OPTION_WHITELISTED_RESOURCE_PATH_PATTERNS + " is invalid", e);
+        String optionWhitelistedSourcePaths = null;
+        if (settings.getOptions().containsKey(OPTION_WHITELISTED_SOURCE_PATH_PATTERNS)) {
+            optionWhitelistedSourcePaths = settings.getOptions().get(OPTION_WHITELISTED_SOURCE_PATH_PATTERNS);
         }
+
+        Collection<String> whitelistedResourcePaths = getPathsFromOption(optionWhitelistedResourcePaths);
+        Collection<String> whitelistedSourcePaths = getPathsFromOption(optionWhitelistedSourcePaths);
+
         try {
             Collection<ContentClassificationMap> maps = new LinkedList<>();
             for (String mapUrl : mapUrls.split("\\s*,\\s*")) {
@@ -96,7 +95,7 @@ public Validator createValidator(@NotNull ValidationContext context, @NotNull Va
                 throw new IllegalArgumentException("At least one valid map must be given!");
             }
             return new AemClassificationValidator(settings.getDefaultSeverity(), new CompositeContentClassificationMap(maps), whitelistedResourcePaths,
-                    getSeverityPerClassification(settings.getOptions().get(OPTION_SEVERITIES_PER_CLASSIFICATION)));
+                    whitelistedSourcePaths, getSeverityPerClassification(settings.getOptions().get(OPTION_SEVERITIES_PER_CLASSIFICATION)));
         } catch (IOException e) {
             throw new IllegalStateException("Could not read from  " + mapUrls, e);
         }
@@ -141,6 +140,22 @@ private static Map<ContentClassification, ValidationMessageSeverity> parseSeveri
         return result;
     }
 
+    private static Collection<String> getPathsFromOption(String optionPaths) {
+        Collection<String> result =
+                Optional.ofNullable(optionPaths)
+                        .map(op -> Arrays.stream(op.split(","))
+                                .map(String::trim)
+                                .collect(Collectors.toList()))
+                        .orElse(Collections.emptyList());
+
+        try {
+            result.stream().forEach(AemClassificationValidatorFactory::validateResourcePathPattern);
+        } catch (IllegalArgumentException e) {
+            throw new IllegalArgumentException("At least one value given in option " + OPTION_WHITELISTED_RESOURCE_PATH_PATTERNS + " is invalid", e);
+        }
+        return result;
+    }
+
     static void validateResourcePathPattern(String resourcePathPattern) throws IllegalArgumentException {
         Matcher matcher = Pattern.compile(resourcePathPattern).matcher("/");
         matcher.matches();