PMDExternalAnnotator has a bug when used with external rulesets. If you want to use an external ruleset to validate Java files, then the ruleset's filename must have "java" in it. Otherwise the ruleset is silently ignored.
This is because of PMDExternalAnnotator#isRuleSetForGivenFile(). It would be better to do this filtering in some other way. Perhaps by removing the filtering altogether and making the annotation rulesets list be per-language, so the user can configure each language independently.