This issue only relates to publicly trusted CAs whose root CA certificate is in web browser trust stores.
PrimeKey has been aware of the recent discussion in Mozilla:
A short description of the issue is that when a Certificate Transparency (CT) pre-certificate is submitted, before a final certificate is issued, and the final certificate is not issued for some reason, OCSP responders are not updated with information about the (non issued) certificate. That means that if your OCSP responder is configured to respond “unknown” to non-issued certificates, this will be the response if someone queries your OCSP responder for the pre-certificate.
The most common reason for the final certificate to not be issued is that the number of SCTs returned from CT log servers are not enough to fulfill the requirements, and issuance of the final certificate is then aborted in order to not deliver an unusable certificate to the subscriber.
It is true that EJBCA, at least up to an including EJBCA 7.3.0, does not store the pre-certificate or a final certificate in case issuance is aborted. OCSP responders are therefore not updated.
There are two different issues discussed in this topic:
- That “unknown” is the response when querying for a pre-certificate where no final certificate has been issued. A proper response would be “good” or “revoked”.
- There is a small time period between when the pre-certificate is issued and submitted to CT logs, and when the final certificate is issued. If someone would query the OCSP responder during this time, the “non-good” response could be cached, and when the final certificate is issued the cached “non-good” OCSP response would be served.
For issue number one there are (at least) two different work-arounds waiting for a feature update of EJBCA:
- Using the “Non existing is revoked” setting on OCSP key bindings will display the correct behavior to the outside.
- Monitoring the server log to know when issuance of a final certificate is aborted when there is not enough SCTs collected from CT logs. When this happens it is possible to insert a revoked record in the database using the command line interface. This event can be clearly identified in the log:
2019-09-09 10:02:03,297 INFO [org.cesecore.certificates.certificatetransparency.CtSubmission] (default task-106) Minimum requirements for SCTs was not was not satisfied for certificate 'tomastest1.primekey.com'.
Insufficient SCTs, minimum is 2, but got 1. 2019-09-09 10:02:03,298 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default task-106) 2019-09-09
tomastest1.primekey.com;ctprecert=true;msg=Failed to submit pre-certificate to the required number of CT logs. The pre-certificate may have been published in some of the logs,
though.;subjectdn=JurisdictionCountry=SE,BusinessCategory=Private Organization,CN=tomastest1.primekey.com,SN=556628-3064,O=PrimeKey Solutions
A CLI command that can be used to insert a certificate in the database, see below under “Importing...”
Issue number two has not been reported to us as an issue in reality yet. There is usually a time period of only maximum a few seconds between sending the pre-certificate to logs and the final certificate being issued. While we are thinking of a solution to this, reports of it happening are welcome. A work-around is to issue a command to flush any external OCSP caches (for example CDN) for the specific certificate issued.
Finding Failed Pre-certificate submissions
Depending on your configuration you may have the information in two logs, or only one.
Database audit log
If you have logging to database enabled, you can search in the database.
Admin UI→Audit Log:
- Module equals Certificate
- Event equals Certificate CT Pre-Cert Submission
- Outcome equals Failure
This will give you a list of all failed CT Pre-Cert submissions (where the result of Pre-Cert submission was not satisfactory so the certificate was not issued). You can click on certificate serial numbers to ensure they are not in the database and you can find the Pre-Certificate in the additional details.
If you have direct access to the database you can also make the same query directly in the database:
select searchDetail1,searchDetail2 from AuditRecordData where eventType='CERT_CTPRECERT_SUBMISSION' and eventStatus='FAILURE';
select searchDetail1,searchDetail2,additionalDetails from AuditRecordData where eventType='CERT_CTPRECERT_SUBMISSION' and eventStatus='FAILURE';
File audit log
If logging to database is not enabled, you have audit logs on file, in server.log, or if you have configured to send logs to a central log server, on the log server. You can search file logs on a linux system with:
grep CERT_CTPRECERT_SUBMISSION;FAILURE *
Where * is replaced with the condition for the files you want to search, for example logs from the last three months.
Once you are satisfied with your search you can send the result to a separate file:
grep "CERT_CTPRECERT_SUBMISSION;FAILURE" server.log* > precertfailures.txt
Importing the Pre-Certificate as Revoked
Extract the username, serial number, certificate profile and base64 encoded certificate data and store the data into a file named by the serial number with:
cat > 21FE23F3894757084E165B5E7A3CD888.b64
where 21FE23F3894757084E165B5E7A3CD888 si the serial number extracted from the log record.
base64 -d 21FE23F3894757084E165B5E7A3CD888.b64 > 21FE23F3894757084E165B5E7A3CD888.der
Convert to PEM format with:
openssl x509 -inform DER -in 21FE23F3894757084E165B5E7A3CD888.der -out 21FE23F3894757084E165B5E7A3CD888.pem
Import as revoked into the CA database:
bin/ejbca.sh ca importcert --username tomastest1.primekey.com-21FE23F3894757084E165B5E7A3CD888 --password pwd --certprofile CTLoggingTLSCert --eeprofile CTLoggingTLSCert --caname "PrimeKey CT Log Test IssuingSub" -a REVOKED -f 21FE23F3894757084E165B5E7A3CD888.pem
You can re-publish this certificate, if you have publishing to OCSP responders for example, from the CA from the Admin UI→Search End Entities→ Search end entity with Certificate SN, clicking View on the certificate and clicking Republish.
You can also choose to import directly into the OCSP responders using the same CLI command.
In case there are many pre-certificates to import, a script can be created to make it faster.