Update Python Netaddr OUI Database

For a small project I needed to validate ~1500 MAC addresses on validity and their vendor Organizational Unique Identifier id (OUI). So a bit of Python scripting was in order.

I used a regular expression for basic MAC address validation, and the netaddr module to check the OUI of the MAC Address. A simple example of the code is shown below

#!/usr/bin/env python3

import netaddr as na
import re

_mac = '88-E9-FE-1F-65-7D'
if re.match('[0-9a-f]{2}([-:]?)[0-9a-f]{2}(\\1[0-9a-f]{2}){4}$', _mac.lower()):
    print(f'{_mac} - {na.EUI(_mac).oui.registration().org}')

Checking this MAC address online gives a normal result.

macvendors.png

Python (or specifically netaddr) not so much, so there some work to be done. The error clearly shows that the OUI for that MAC address is not registered (in netaddr’s local database).

netaddr-error.png

The problem is that the OUI ‘database’ from netaddr is (extremely) out-dated, so recently assigned OUI’s are not available, and result in Python script errors.

Unfortunatelly, the netaddr documentation doesn’t give any hint on how to update this database. Some searching on the local filesystem showed that there is a oui.txt file within the directory structure of netaddr (which in my case can be seen in the error shown above).

The latest oui.txt (~0.5MB larger than the netaddr version) can be downloaded @ IEEE (the organization were hardware vendors can request new OUI’s). The file location is: http://standards-oui.ieee.org/oui.txt.
I downloaded the file and replaced the original netaddr version. Running the code again gave no solution, since I got the same error. So back to the drawing board.

In the same directory as the oui.txt is a file called oui.idx. This file contains the decimal value of the OUI, and an offset. It turns out that the netaddr codeused this idx file to quickly skip to the actual vendor information in the oui.txt file. And since my idx file was based on the old oui.txt the vendor could still not be found.

The idx file cannot be found on the internet. It’s not something IEEE provides. It’s a file generated from the information in the oui.txt file.

Solution: In the netaddr directory where the oui.txt and oui.idx is located is a ieee.py script. Run that script, and it creates a new idx file based on the oui.txt file in that directory (as shown in the following example).

myhost:eui myname$ pwd
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/netaddr/eui
myhost:eui myname$ ls -la
total 12856
drwxr-xr-x   9 myname  admin      288 Jan 26 13:37 .
drwxr-xr-x  11 myname  admin      352 Jan 26 13:37 ..
-rw-r--r--   1 myname  admin    24990 Jan 26 13:37 __init__.py
drwxr-xr-x   4 myname  admin      128 Jan 26 13:37 __pycache__
-rw-r--r--   1 myname  admin    95467 Jan 26 13:37 iab.idx
-rw-r--r--   1 myname  admin  2453271 Jan 26 13:37 iab.txt
-rw-r--r--   1 myname  admin     9500 Jan 26 13:37 ieee.py
-rw-r--r--   1 myname  admin   419098 Jan 26 13:37 oui.idx
-rw-r--r--   1 myname  admin  3566144 Jan 26 13:37 oui.txt

myhost:eui myname$ curl http://standards-oui.ieee.org/oui.txt --output oui.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4039k  100 4039k    0     0   370k      0  0:00:10  0:00:10 --:--:--  437k

myhost:eui myname$ python3 ieee.py 
myhost:eui myname$ ls -la
total 14336
drwxr-xr-x   9 myname  admin      288 Jan 26 13:37 .
drwxr-xr-x  11 myname  admin      352 Jan 26 13:37 ..
-rw-r--r--   1 myname  admin    24990 Jan 26 13:37 __init__.py
drwxr-xr-x   4 myname  admin      128 Jan 26 13:37 __pycache__
-rw-r--r--   1 myname  admin    95467 May 10 12:14 iab.idx
-rw-r--r--   1 myname  admin  2453271 Jan 26 13:37 iab.txt
-rw-r--r--   1 myname  admin     9500 Jan 26 13:37 ieee.py
-rw-r--r--   1 myname  admin   485973 May 10 12:14 oui.idx
-rw-r--r--   1 myname  admin  4136058 May 10 12:14 oui.txt
myhost:eui myname$ 

After that the output of my script was the following:

/usr/local/bin/python3.7 /Volumes/Python_Scripts/test.py
88-E9-FE-1F-65-7D - Apple, Inc.

Process finished with exit code 0
Posted on May 10, 2019 and filed under Programming, Annoying.