Expected failure modes for saml.oktadev.com SSO tests

I’m testing our new Okta SSO integration for IdP-initiated authentication and am confused by three similar failures I’m encountering with the extended security validation using saml.oktadev.com. Specifically these test cases:

  • SAMLResponse that removes NotOnOrAfter from saml:SubjectConfirmationData
  • SAMLResponse that removes NotOnOrAfter from saml:Conditions
  • SAMLResponse that removes NotBefore and NotOnOrAfter from saml:Conditions

…all fail because my integration accepts the modified response. I noted that the modified responses all remove the assertion-level signature and provide a new response-level signature.

I was assuming that the expected failure mode would be due to an invalid signature, but validation by multiple approaches (OpenSaml SignatureValidator, XmlSignature.validate()) all indicate that the new signatures are valid for the SAML IdP Metadata for idp.oktadev.com key.

Note that our integration DOES correctly reject the other test cases in which some value is modified (eg. NameID). Test cases on modified Destination, Issuer, and Audience tag contents are also correctly rejected, though due to explicit checks against expected values.

Is there some other failure mode that these test cases are exercising which I’m not checking, eg. requiring time constraints on Conditions or SubjectConfirmationData? Simple enough to enforce that constraint, but these attributes are optional according to the SAML 2.0 specification. What should I be checking for?

FWIW, this is a Java 8 project using opensaml 2.6.4