Overlays + MDB == OpenLDAP Fun!

Recently, I was given the task/honor of setting up an OpenLDAP server to be used for the enterprise. In the past, I have set up a few OpenLDAP servers – some built from source; others from packages. This time, it was different. Back in early May, I went to UDS in Oakland, California. When I was there, I sat in on a track where Howard Chu presented MDB: A Memory-Mapped Database and Backend for OpenLDAP. Every since then, I was inspired to deploy an OpenLDAP server utilizing MDB – which also gave me a chance to play around with Overlays. This blog will give a breakdown of what steps were taken to deploy an OpenLDAP server from source, using MDB backends – utilizing a few overlays.

Prerequisites

First off, we need to get the source code for the latest version of OpenLDAP. Go to the Downloads link on the OpenLDAP home page. Once there, grab any one of the links there and use wget/curl to download the tar-gzipped file.

wget ftp://ftp.OpenLDAP.org/pub/OpenLDAP/openldap-release/openldap-2.4.31.tgz

Once the tar-gzipped file is downloaded, untar it:

tar -zxvf openldap-2.4.31.tgz

The prerequisites for OpenLDAP are listed on the prerequisites page.

The prerequisites I used were as follows:

  • gcc
  • glibc-devel
  • libtool-ltdl
  • db4-devel
  • openssl-devel
  • unixODBC-devel

These were used based upon the features to be utilized with OpenLDAP. To learn more about what features can be used with OpenLDAP, use the configure command to get some help:

cd openldap-2.4.31; ./configure --help

Configure, Build, Install

After determining what features will be used with OpenLDAP, its time to configure the build. Here is an example of running the configure command:

./configure --prefix=/opt/openldap --enable-debug=yes --enable-syslog --enable-dynamic --enable-slapd --enable-dynacl --enable-cleartext --enable-spasswd --enable-modules --enable-rewrite --enable-rlookups --enable-bdb --enable-dnssrv=mod --enable-hdb --enable-mdb --enable-monitor --enable-overlays --with-cyrus-sasl --with-threads --with-tls=openssl CC="gcc" LDFLAGS="-L/usr/lib64/sasl2" CPPFLAGS="-I/usr/include/sasl"

After that has ran successfully, all that is left is to use make to complete the installation. Run the following commands:

make depends
make
make test (Here’s the time to go grab some dinner and watch the NBA playoffs)

If all goes well, all that is left is to install:

make install

Setup – OLC and MDB

Create a user that will run the slapd process:

useradd -m -U -c "OpenLDAP User" -s /bin/bash openldap

Create the directory where the main database will be running:

mkdir /opt/openldap-2.4.31/var/openldap-data/main; chown -R openldap:openldap /opt/openldap-2.4.31/var/openldap-data/main

Create passwords for admin user for config database, and Directory Manager for main directory by using slappasswd:

/opt/openldap-2.4.31/sbin/slappasswd -h {SSHA} (Run this for each password to be created.)

Now we need to configure the server. We will be using the on-line configuration (OLC) of slapd (instead of the old configuration way – slapd.conf. OLC provides close to zero down-time configuration, cn=config and slapd.d directory. For more information concerning usage of cn=config, please refer to Configuring slapd in the online OpenLDAP Administrator’s Guide.

Not only will OLC be used, but MDB will be set up for the main database. For more information about MDB, please refer to the whitepaper, MDB: A Memory-Mapped Database and Backend for OpenLDAP.

For the setup, we need to create a slapd.conf, with the following information:


#######################################################################
# Config database definitions
#######################################################################
dn: cn=config
objectClass: olcGlobal
cn: config
olcArgsFile: /opt/openldap-2.4.31/var/run/slapd.args
olcPidFile: /opt/openldap-2.4.31/var/run/slapd.pid

dn: olcDatabase=config,cn=config
olcDatabase: config
olcRootPW: {SSHA}xxxxxxxxxxxxxxxxxxxxxx
olcAccess: to * by * none
olcLogLevel: -1

dn: cn=schema,cn=config
objectClass: olcSchemaConfig
cn: schema

include: file:///opt/openldap-2.4.31/etc/openldap/schema/core.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/collective.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/corba.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/cosine.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/duaconf.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/dyngroup.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/inetorgperson.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/misc.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/nis.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/openldap.ldif
include: file:///opt/openldap-2.4.31/etc/openldap/schema/ppolicy.ldif

# Frontend settings
#
dn: olcDatabase=frontend,cn=config
objectClass: olcDatabaseConfig
olcDatabase: frontend
olcAccess: to dn.base="" by * read
olcAccess: to dn.base="cn=Subschema" by * read
olcAccess: to *
by self write
by users read
by anonymous auth

#######################################################################
# MDB database definitions
#######################################################################
#
dn: olcDatabase=mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: mdb
olcSuffix: dc=example,dc=com
olcRootDN: cn=Directory Manager,dc=example,dc=com
olcRootPW: {SSHA}xxxxxxxxxxxxxxxxxxxxxx
olcDbDirectory: /opt/openldap-2.4.31/var/openldap-data/main
olcDbIndex: objectClass eq
olcAccess: to attrs=userPassword by dn="cn=Directory Manager,dc=example,
dc=com" write by anonymous auth by self write by * none
olcAccess: to attrs=shadowLastChange by self write by * read
olcAccess: to dn.base="" by * read
olcAccess: to * by dn="cn=Directory Manager,dc=example,dc=com" write by
* read
olcDbMaxReaders: 0
olcDbMode: 0600
olcDbSearchStack: 16
olcDbMaxSize: 4294967296
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 15
olcReadOnly: FALSE
olcSyncUseSubentry: FALSE
olcMonitoring: TRUE
olcDbNoSync: FALSE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync

Schema Information

In the schema section, the following LDIFs were included:

  • collective => Collective attributes
  • corba => Corba Object
  • core => OpenLDAP “core”
  • cosine => COSINE Pilot
  • duaconf => Client Configuration
  • dyngroup => Dynamic Group
  • inetorgperson => InetOrgPerson
  • misc => Miscellaneous Schema
  • nis => Network Information Service
  • openldap => OpenLDAP Project
  • ppolicy => Password Policy Schema

For more information about the schema configuration, please refer to the following links:

Config Information

For more information concerning using OLC (cn=config), please check out these links:

After the slapd.conf file is set up, use slaptest to create the slapd.d directory to finish out the configuration:

/opt/openldap-2.4.31/sbin/slaptest -f /opt/openldap-2.4.31/etc/openldap/slapd.conf -F /opt/openldap-2.4.31/etc/openldap/slapd.d

Ready to Start the OpenLDAP Server

Now its time to start the OpenLDAP server. To start OpenLDAP, run the following command:

/opt/openldap-2.4.31/libexec/slapd -F /opt/openldap-2.4.31/etc/openldap/slapd.d/ -h "ldap:/// ldapi:/// ldaps:///" -d -1 -u openldap -g openldap

If all is well, the overlays can now be added.

Overlays and Additional Databases

Rewrite Overlay

The Rewrite Overlay uses the slapo-rwm overlay and the relay proxy backend. This overlay/backend combo will allow “dc=example,dc=com” or “o=example” to be used by ldap tools, such as ldapsearch.

To implement this, create an LDIF file with the following information


# cat rewrite-overlay.ldif

dn: olcDatabase={2}relay,cn=config
objectClass: olcDatabaseConfig
objectClass: olcRelayConfig
olcDatabase: {2}relay
olcSuffix: o=example
olcRelay: dc=example,dc=com

dn: olcOverlay={0}rwm,olcDatabase={2}relay,cn=config
objectClass: olcOverlayConfig
objectClass: olcRwmConfig
olcOverlay: {0}rwm
olcRwmRewrite: {0}rwm-rewriteEngine "on"
olcRwmRewrite: {1}rwm-suffixmassage "dc=example,dc=com"

Once that file is created, add the ldif to the cn=config database:

ldapadd -x -D cn=admin,cn=config -w -f rewrite-overlay.ldif

When data is added to dc=example,dc=com, searches can be done using the base “dc=example,dc=com” or “o=example”.

Monitoring

Setting up the monitoring database gives the ability to monitor the status and gain statistics of the OpenLDAP server through ldapsearch.

To set up the monitoring database, create the following LDIF:


# cat monitor.ldif

dn: olcDatabase={3}Monitor,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMonitorConfig
olcDatabase: {3}Monitor
olcAccess: {0}to * by dn="cn=Directory Manager,dc=example,dc=com" read

After this file has been created, use ldapadd to add it to the config database:

ldapadd -x -D cn=admin,cn=config -w -f monitor.ldif

Now, statistics can be gathered by the Directory Manager searching against the monitor database:

ldapsearch -x -D 'cn=Directory Manager,dc=example,dc=com' -w -b 'cn=Monitor' -s base 1.1

Auditing Overlay

The auditing overlay is great for security audits. For a given database, an LDIF file is created of all ldap operations completed against the database. This is also helpful for disaster recovery as well.

Since the config database and the main mdb database (dc=example,dc=com) will be accessed the most, lets create an audit overlay for them. To do so, add the following information to an LDIF:


# cat auditlog.ldif

dn: olcOverlay={0}auditlog,olcDatabase={0}config,cn=config
objectClass: olcOverlayConfig
objectClass: olcAuditlogConfig
olcOverlay: {0}auditlog
olcAuditlogFile: /var/log/ldap/auditlog-config.ldif

dn: olcOverlay={0}auditlog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAuditlogConfig
olcOverlay: {0}auditlog
olcAuditlogFile: /var/log/ldap/auditlog-mdb.ldif

To apply the overlay, use ldapadd on the config database:

ldapadd -x -D cn=admin,cn=config -w -f auditlog.ldif

There will now be an ldif file for each database under /var/log/ldap directory.

Access Log Overlay

The accesslog overlay is another overlay that is implemented with data storage database. This overlay allows the Directory Manager to see all successful ldap modifies, deletes, searches, adds against the main directory (dc=example,dc=com). This information can be accessed using ldapsearch. There is the ability to implement log purging as well. In this setup, any logs older than 7 days will be deleted. Since the database will be stored under /opt/openldap-2.4.31/var/openldap-data/access, we need to make sure and create that directory, and set the ownership to the openldap user:

mkdir /opt/openldap-2.4.31/var/openldap-data/access;chown -R openldap:openldap /opt/openldap-2.4.31/var/openldap-data/access

To set up the accesslog overlay, create the following LDIF:


# cat access-log.ldif
dn: olcDatabase={4}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {4}mdb
olcDbDirectory: /opt/openldap-2.4.31/var/openldap-data/access
olcSuffix: cn=log
olcDbIndex: reqStart eq
olcDbMaxSize: 1073741824
olcDbMode: 0600
olcAccess: {0}to * by dn="cn=Directory Manager,dc=example,dc=com" read

dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=log
olcAccessLogOps: all
olcAccessLogPurge: 7+00:00 1+00:00
olcAccessLogSuccess: TRUE
olcAccessLogOld: (objectclass=person)

Logging will be done for any action done by any person in the main directory (dc=example,dc=com). Now its time to add the ldif to the config database:

ldapadd -x -D cn=admin,cn=config -w -f access-log.ldif

Logging can now be accessed by the Directory Manager by using ldapsearch:


# ldapsearch -D "cn=Directory Manager,dc=example,dc=com" -w -b cn=log

# extended LDIF
#
# LDAPv3
# base with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# log
dn: cn=log
objectClass: auditContainer
cn: log

# 20120607233002.000000Z, log
dn: reqStart=20120607233002.000000Z,cn=log
objectClass: auditBind
reqStart: 20120607233002.000000Z
reqEnd: 20120607233002.000001Z
reqType: bind
reqSession: 1071
reqAuthzID:
reqDN: cn=Directory Manager,dc=example,dc=com
reqResult: 0
reqVersion: 3
reqMethod: SIMPLE
........

Thats pretty much it. OpenLDAP using MDB, with some pretty cool overlays. What more could you ask for?

Overlays + MDB == OpenLDAP Fun!

9 thoughts on “Overlays + MDB == OpenLDAP Fun!

  1. paco says:

    Hello hspencer77,

    First, thank you for the clear documentation.

    I compiled openldap. And I tried to insert my slapd.conf (for my use) and your slapd.conf but i have the same error:

    “line 4: unknown directive outside backend info and database definitions.
    slaptest: bad configuration directory!”

    If you know this error…i search i search but i don’t find to resolve this problem.

    Thank you!

    1. hspencer77 says:

      Hi paco,

      The slapd.conf that you have, if you run the following command, what result to you get:

      /opt/openldap-2.4.31/sbin/slaptest -f slapd.conf

      This will let us know if there is a format issue with your slapd.conf.

      Hope this helps.

      Regards,

      -H

  2. paco says:

    It’s the same result…

    Where is the problem? In the first block in slapd.conf file?:
    “dn: cn=config
    objectClass: olcGlobal
    cn: config
    olcArgsFile: /opt/openldap/var/run/slapd/slapd.args
    olcPidFile: /opt/openldap/var/run/slapd/slapd.pid”

    ….. or in other part of the file?

    Thanks

    1. Definitely something up with your slapd.conf. Is the directory /opt/openldap/var/run/slapd created? What –prefix option did you use when you ran the ./configure?

  3. paco says:

    ./configure –prefix=/opt/openldap –enable-debug=yes –enable-syslog –enable-dynamic –enable-slapd –enable-dynacl –enable-cleartext –enable-spasswd –enable-modules –enable-rewrite –enable-rlookups –enable-bdb –enable-dnssrv=mod –enable-hdb –enable-mdb –enable-monitor –enable-overlays –with-cyrus-sasl –with-threads –with-tls=openssl CC=”gcc”

    A part of my slapd.conf:
    dn: cn=config
    objectClass: olcGlobal
    cn: config
    olcArgsFile: /opt/openldap/var/run/slapd.args
    olcPidFile: /opt/openldap/var/run/slapd.pid
    olcToolThreads: 1
    olcThreads: 16
    olcSizeLimit: 0

    dn: olcDatabase={0}config,cn=config
    olcDatabase: config
    olcRootPW: {crypt}***************
    olcAccess: to * by * none
    olcLogLevel: -1

    dn: cn=schema,cn=config
    objectClass: olcSchemaConfig
    cn: schema

    include: file:///opt/openldap/etc/openldap/schema/core.ldif
    include: file:///opt/openldap/etc/openldap/schema/cosine.ldif
    include: file:///opt/openldap/etc/openldap/schema/inetorgperson.ldif

    # Frontend settings
    dn: olcDatabase={-1}frontend,cn=config
    objectClass: olcDatabaseConfig
    olcDatabase: frontend
    olcSizeLimit: 500
    olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external
    ,cn=auth manage by * break
    olcAccess: {1}to dn.exact=”” by * read
    olcAccess: {2}to dn.base=”cn=Subschema” by * read

    1. Hey paco,

      Let test it out. Everything looks right. Additionally, I am not sure if you have tried the #openldap IRC channel on irc.freenode.net. I will respond as quick as I can, but you may get a quicker response there.

      Cheers,

      -H

Leave a comment