OpenX can be used to setup adnetwork just like any other private adnetwork like double click or Yahoo publisher network. There are lots of features available as plugins and mods from sites like www.openxmods.com and www.openxplugins.com using which you can go with basic installation of OpenX and start installing whichever features you would require for your adnetwork.
There are lots of adnetwork sites successfully running using OpenX as its backend.
OpenXservices.com provides custom solutions for advertising agencies to setup its own adnetwork.