Massively collecting CRL and OCSP information

November 3, 2011  |  Jaime Blasco

As part of the IP reputation project we are writing a small engine to avoid false positives and whitelisting some common ips/networks.

Usually when you execute a binary on a sandbox and the executable file has been signed, you receive a lot connections to the servers hosting the Certificate Revocation Lists (CRL) and the Online Certificate Status Protocol (OCSP).

To avoid processing this ips, we use some scripts to parse and extract the most used CRL and OCSP servers extracting this information from certificates.

Right now we are using the EFF SSL Observatory dataset and also the Alexa Top 1M list.

Let’s begin with the SSL Observatory database. Once we have the Mysql database ready, execute the following sql query to extract the OCSP URIs:

select `X509v3 extensions:Authority Information Access:OCSP - URI` as ocsp,count(*) as total INTO OUTFILE ‘/tmp/ocsp.csv’ FIELDS TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY ‘“’ LINES TERMINATED BY ‘ ’ from all_certs where `X509v3 extensions:Authority Information Access:OCSP - URI` is not NULL group by ocsp order by total desc;

Then you can use this script [no longer available] to parse the file:

tor@tor-VirtualBox:~$ python /tmp/ocsp.csv


We can do the same for CRL entries using this other script [no longer available]:

tor@tor-VirtualBox:~$ python /tmp/crls.csv


The other script [no longer available] I want to share parses the Alexa TOP 1M list, extracts the SSL certificate if https is supported and then extracts the OCSP/CRL URIS:

jaimes-MacBook-Pro:PKIS jaime$ python2.7


So mixing the outputs we have a list of the most used PKI servers that we can classify as normal activity.

Share this with others

Get price Free trial