SPNEGO authentication

I have written previously about authenticating against Active Directory using LDAP, and I will probably be writing in the future about authenticating using a username and password against AD's Kerberos, but I have not mentioned the original reason for my interest in the grinding toil of groveling through Microsoft authentication protocols.

The Simple and Protected GSSAPI Negotiation scheme and its application to HTTP provide a nice solution to single sign-on to web applications in a Kerberos-based authentication environment. It works basically like this:

1. The browser connects to the server and makes a request.


2. The server determines that the client is not authorized and the request has no authorization information in it, and denies the request with a 401 status code and a "WWW-Authenticate" header of "Negotiate".


3. The browser identifies that it has Kerberos authentication information available (a cached TGT, for example). It requests a ticket for the server from the ticket-grating authority and re-sends the original request with the ticket (wrapped in some GSS-API nonsense and base-64 encoded) in the "Authorization" header.


4. The server recovers the ticket and validates it, based on its pre-authentication with the same Kerberos authority. Assuming the ticket validates, the server replies successfully to the request, possibly adding some authentication-negotiation-related information to the response.

There are some complications; the negotiation may take more than one round, etc., but for the most-common, successful case, it is a straightforward, relatively lightweight protocol that does not require (nor allow, for that matter) any user effort. So, how do you do this for a Java web application? Well, I have a servlet filter that implements the authentication protocol, and I would like to describe it here.

A servlet filter implementation requires three methods, and destroy() is currently unused. That leaves init(FilterConfig) and doFilter(ServletRequest, ServletResponse, FilterChain). init() handles the configuration, based on the configuration in the web app's web.xml file.

The first part of init recovers four pieces of information from the configuration:

• The service's Service Principal Name (SPN).


• The location of the key tab file containing the SPN's private key.


• Whether or not some debugging info should be printed.


• A regular expression describing resources such as images or freely-accessible pages which should not require authentication.

    final String spn = filterConfig.getInitParameter(SERVICE_PRINCIPAL_NAME_PARAMETER);
    final String keyTab = filterConfig.getInitParameter(SERVICE_KEYTAB_FILE_PARAMETER);
    if (spn == null || keyTab == null)
    {
      throw new ServletException( String.format("both %s and %s must be set", SERVICE_PRINCIPAL_NAME_PARAMETER, SERVICE_KEYTAB_FILE_PARAMETER) );
    }
    String passExpression = filterConfig.getInitParameter(NON_AUTHENTICATED_REGEX_PATTERN);
    if (passExpression != null)
    {
      try
      {
        passPattern = Pattern.compile(passExpression);
      }
      catch (PatternSyntaxException e)
      {
        throw new ServletException( String.format("invalid %s expression: %s", NON_AUTHENTICATED_REGEX_PATTERN, passExpression), e);
      }
    }
    final String debugParameter = filterConfig.getInitParameter(DEBUG_PARAMETER);
    final boolean debug = (debugParameter != null && debugParameter.equalsIgnoreCase("true"));

Given these four items (actually, just the first three), I can create a Java standard Subject object which can validate incoming tickets.

    final Configuration loginConfig = new KerberosLoginConfiguration(spn, keyTab, debug);
    final Set principals = new HashSet(1);
    principals.add(new KerberosPrincipal(spn));
    final Subject subject = new Subject(false, principals, Collections.EMPTY_SET, Collections.EMPTY_SET);
    try
    {
      final LoginContext loginContext = new LoginContext("", subject, null, loginConfig);
      loginContext.login();
      serviceSubject = loginContext.getSubject();
    }
    catch (LoginException e)
    {
      throw new ServletException( String.format("cannot login to %s with %s", spn, keyTab), e);
    }

(This implementation is based on an example from Spring Security; my original implementation required several much uglier custom classes.)

The SPN, keytab file path, and debug flag are encoded in a KerberosLoginConfiguration which is used to conceptually pre-authenticate the server's resulting serviceSubject. The key is the keytab file, which contains the private key for the SPN, which is associated with a Kerberos account and exported from Active Directory. This keytab file should be should be protected in much the same way as an SSL key.

The KerberosLoginConfiguration is

public class KerberosLoginConfiguration extends Configuration
{
  private final Map options = new HashMap();

  public KerberosLoginConfiguration(String spn, String keyTabFile, boolean debug)
  {
    options.put("useKeyTab", "true");
    options.put("storeKey", "true");
    options.put("doNotPrompt", "true");
    options.put("isInitiator", "false");
    options.put("keyTab", keyTabFile);
    options.put("principal", spn);
    options.put("debug", Boolean.toString(debug));
  }

  @Override
  public AppConfigurationEntry[] getAppConfigurationEntry(String name)
  {
    final AppConfigurationEntry[] entries = {
        new AppConfigurationEntry(BaseKerberosSpnegoFilter.KERBEROS_LOGIN_MODULE, AppConfigurationEntry.LoginModuleControlFlag.REQUIRED, options)
    };
    return entries;
  }
}

(All of this is using the hideous JAAS interfaces. I'm sorry.)

The Configuration contains a number of elements needed to configure a com.sun.security.auth.module.Krb5LoginModule, the value of KERBEROS_LOGIN_MODULE, to use the SPN and keytab file, and ultimately to handle the SPNEGO negotiation.

Once I have the serviceSubject, which is an attribute of the Filter implementation, I am off to the races. The actual filter starts by managing some necessary trivia.

• First, it checks if the request is trying to sneak information in using one of the mechanisms that I use to pass authentication on to the application. Silly, yes, but, well, there you are.


• Then, it checks whether it should just pass the request on. See passRequest below.


• Finally, it picks out the Authorization header and parses the result. If the header does not exist, it returns the result which starts the negotiation.

    final HttpServletRequest request = (HttpServletRequest) req;
    final HttpServletResponse response = (HttpServletResponse) resp;

    if (request.getHeader(KerberosAuthenticatedHttpRequest.AUTHENTICATED_USER_HEADER) != null)
    {
      // Don't come back now, ya hear?
      log.warn(String.format("%s header set in request", KerberosAuthenticatedHttpRequest.AUTHENTICATED_USER_HEADER));
      response.sendError(HttpServletResponse.SC_BAD_REQUEST, String.format("%s header set in request", KerberosAuthenticatedHttpRequest.AUTHENTICATED_USER_HEADER));
      return;
    }
    if (passRequest(request))
    {
      chain.doFilter(request, response);
      return;
    }
    final String header = request.getHeader(AUTHORIZATION_HEADER);
    if (header == null)
    {
      response.setHeader(WWW_AUTHENTICATE_HEADER, NEGOTIATE_HEADER_VALUE);
      response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
      return;
    }
    final String[] elements = header.split(" +");
    if (! elements[0].equalsIgnoreCase("Negotiate"))
    {
      response.sendError(HttpServletResponse.SC_BAD_REQUEST, String.format("unknown authentication mechanism '%s'", header));
      return;
    }
    else if (elements.length != 2 || elements[1] == null)
    {
      response.sendError(HttpServletResponse.SC_BAD_REQUEST, String.format("unable to get SPNEGO ticket in '%s'", header));
      return;
    }

Once the preliminaries are handled, the ticket validation can begin.

    try
    {
      final SpnegoTicketValidator validator = Subject.doAs(serviceSubject, new SpnegoTicketValidator(elements[1]));
      if (validator.getNextToken() != null)
      {
        response.setHeader(WWW_AUTHENTICATE_HEADER, String.format("%s %s", NEGOTIATE_HEADER_VALUE, validator.getNextToken()));
      }
      final GSSContext context = validator.getContext();
      if (! context.isEstablished())
      {
        response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
        return;
      }
      // Authentication accepted; continue with request
      chain.doFilter(new KerberosAuthenticatedHttpRequest(request, context), response);
      // Dispose of the GSSContext, freeing any resources.
      try
      {
        context.dispose();
      }
      catch (GSSException e)
      {
        System.err.println("error disposing of GSSContext");
        e.printStackTrace(System.err);
      }
    }
    catch (PrivilegedActionException e)
    {
      response.sendError(HttpServletResponse.SC_UNAUTHORIZED, "authorization failed");
    }

Actually, the third line and the SpnegoTicketValidator object handles the important parts. The rest is more details. The ticket negotiation may have another token to pass back to the client, or the negotiation may not be complete; these two situations must be handled before the request is passed on to the client. Once the request is handled, the GSSContext should be free'ed. (And I just noticed a bug; the context should be disposed if the negotiation is not complete. Gotta fix that.)

The request is wrapped by a KerberosAuthenticatedHttpRequest, to pass the authentication information to the application. The important part is the SpnegoTicketValidator.

public class SpnegoTicketValidator implements PrivilegedExceptionAction
{
  final private byte[] token;
  private GSSContext context;
  private byte[] nextToken;

  public SpnegoTicketValidator(String ticket)
  {
    this.token = Base64.decodeBase64(ticket.getBytes());
  }

  @Override
  public SpnegoTicketValidator run() throws Exception
  {
    context = GSSManager.getInstance().createContext((GSSCredential) null);
    nextToken = context.acceptSecContext(token, 0, token.length);
    return this;
  }

  public GSSContext getContext()
  {
    return context;
  }

  public String getNextToken()
  {
    return (nextToken == null) ? null : new String( Base64.encodeBase64(nextToken) );
  }
}

The key is the cryptographic context; after it accepts the token, assuming the negotiation is complete, it contains the information about the user on the other end of the connection. This class also hangs onto the possible next token to return to the client.

There remains only a few loose ends. One is the passRequest method, which I wrote in this way to allow subclasses to override it with more complicated pass/authenticate decisions.

  protected boolean passRequest(HttpServletRequest request)
  {
    if (passPattern == null)
    {
      return false;
    }
    StringBuffer url = request.getRequestURL();
    String queryString = request.getQueryString();
    if (queryString != null)
    {
      url.append('?').append(queryString);
    }
    return passPattern.matcher(url).matches();
  }

The other loose end in the KerberosAuthenticatedHttpRequest, which I am too tired to reproduce at the moment. The highlights are:

• context.getSrcName().toString() provides the user's identity in the form of "username@domain".


• new KerberosPrincipal( context.getSrcName().toString() ) creates an implementation of the Principal interface.

Whatever happened to symbolic links?

To tell the truth, I am relatively new to Java programming. Prior to my current job, I had only briefly used Java in anger, while working at IBM, and in fact even there I had much more experience with JNI than with actual Java code.

One thing that always irritated me when I looked at the language was that significant chunks of functionality were missing from this supposed system programming language. For example, how do you get access to environment variables? Sure, there was System.getenv, but until relatively recently that was deprecated and threw an exception. Heck, Runtime.exec allows you to provide an environment to a sub-process. Even the hoary "system dependent" excuse seems unreasonable, since beaucoup other portable languages seemed to be capable of doing something useful.

Another example is symbolic links. (If you do not know what they are, you can find the definition elsewhere. I'm bitter.) Symbolic links are one of the most useful tools in the eternal war between hideous hacks and non-functionality. (Ok, sometimes they're not hacks. Did I mention I am bitter?) But Java (as of 6) does not recognize them, cannot create them, and in at least one case does the most massive wrong thing with them.

Suppose, for example, you have a directory that you wish to recursively delete. Suppose further that this directory contains a symbolic link to another directory, which you do not wish to delete. Now, most deletey things like rm handle this situation correctly: they delete the symbolic link itself, but leave the target directory untouched. Naive Java code, on the other hand, only recognizes the existence of files and directories and will cheerfully follow the link and blow away the contents of the target directory.

How do I know this? I have a web application that needs to publish content files from a stable directory outside of the exploded war file, and I am too lazy to modify the application's code to correctly look outside its own directory. This situation is the poster child of symlink uses. But what happens when you undeploy the web app? The exploded war file's directory is recursively deleted by naive Java code, at least in both Tomcat and the SpringSource dm Server.

The original, recursive delete code looks something like:

private static boolean doRecursiveDelete(File root) {
  if (root.exists()) {
    if (root.isDirectory()) {
      File[] children = root.listFiles();
      if (children != null) {
        for (File file : children) {
          doRecursiveDelete(file);
        }
      }
    }
    return root.delete();
  }
  return false;
}

That method cheerfully follows symbolic links, since isDirectory is true for a link to a directory.

Fortunately, I found a patch from July, 2008 by Michael Bailey that attempts to fix the problem for Tomcat, and that seems to have a positive review. (On the other hand, I found a similar Tomcat bug and patch from August, 2009, that seems to be labeled WONTFIX.)

I created a patch for the SpringSource dm Server that we are using and life seems to be better. I also reported a bug and included the patch.

These patches work...strangely. Java does not recognize anything but files and directories, but it does let you get the canonical path to a file system object. If the canonical path of an object differs from the canonical path of its parent directory plus the object's name, there might be a symlink involved:

  String path1 = file.getAbsoluteFile().getParentFile().getCanonicalPath() + File.separatorChar + file.getName();
  String path2 = file.getAbsoluteFile().getCanonicalPath();
  return !(path1.equals(path2));

Furthermore, you may be wondering how to create a symlink in Java? One cow-orker (Hi, Del!) suggested exec'ing ln -s; I did not think of that since I like JNI too much:

JNIEXPORT void JNICALL
Java_util_Symlink_symlink(JNIEnv *env, jclass cls, jstring oldPath, jstring newPath)
{
  const jbyte *old = (*env)->GetStringUTFChars(env, oldPath, NULL);
  if (old == NULL) {
    jclass exCls = (*env)->FindClass(env, "java/io/IOException");
    if (exCls != NULL) {
      (*env)->ThrowNew(env, exCls, "cannot access oldPath");
    }
    (*env)->DeleteLocalRef(env, exCls);
    return;
  }
  const jbyte *news = (*env)->GetStringUTFChars(env, newPath, NULL);
  if (news == NULL) {
    (*env)->ReleaseStringUTFChars(env, oldPath, old);
    jclass exCls = (*env)->FindClass(env, "java/io/IOException");
    if (exCls != NULL) {
      (*env)->ThrowNew(env, exCls, "cannot access newPath");
    }
    (*env)->DeleteLocalRef(env, exCls);
    return;
  }
  int rc = symlink(old, news);
  (*env)->ReleaseStringUTFChars(env, oldPath, old);
  (*env)->ReleaseStringUTFChars(env, newPath, news);
  if (rc) {
    jclass exCls = (*env)->FindClass(env, "java/io/IOException");
    if (exCls != NULL) {
      (*env)->ThrowNew(env, exCls, strerror(errno));
    }
    (*env)->DeleteLocalRef(env, exCls);
    return;
  }
}

As for Java and I, well, I am learning to adapt. Java is not the worst programming language I have used. And if I am parsing the Google results correctly, symlinks will be handled in Java 7.

[Edit: Why do I keep wanting to spell "canonical" with three n's?]

Brilliant hack

A cow-orker, Del Johnson, just pointed me to ditaa, Diagrams through Ascii Art. Yes, it creates diagrams from ASCII art. Sweeeeeeet, eh?

I grabbed the TCP header diagram from RFC 793, and with very little effort produced:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | | | | | | | | | | |1| | | | | | | | | |2| | | | | | | | | |3| |
   |0|1|2|3|4|5|6|7|8|9|0|1|2|3|4|5|6|7|8|9|0|1|2|3|4|5|6|7|8|9|0|1|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Source Port          |       Destination Port        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Acknowledgment Number                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data |           |U|A|P|R|S|F|                               |
   | Offset| Reserved  |R|C|S|S|Y|I|            Window             |
   |       |           |G|K|H|T|N|N|                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Checksum            |         Urgent Pointer        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Options                    |    Padding    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             data                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

(I added boxes around the bit-numbers at the top.) Using ditaa, that generates:

Yes, the bit-width labels are kind of mangled, and I do not know why some of the other labels are in a different font, but for version 0.6b....daaaang.
All those years spent learning to draw ASCII diagrams have not gone to waste!
Another bit of wisdom from Del: The collective form of "minutia" is "manure."

n log n is like a cheese log

Johnny's Algorithms Homework

I have no real comment.

Authenticating against an Active Directory Server, pt. 5

• Authenticating against an Active Directory Server, pt. 1: Introduction


• Authenticating against an Active Directory Server, pt. 2: ASN.1


• Authenticating against an Active Directory server, pt. 3: DNS lookups


• Authenticating against an Active Directory Server, pt. 4: LDAP Ping request

With this post, I am getting to the actual punch line of the series, authenticating a username and password to a dynamically identified Active Directory server. Further, I just made a realization: this last post contains the entire Internet. The whole ball of wax is here, except perhaps for spam. A protocol defined to run on TCP but transported instead by UDP. ASN.1 and LDAP, the last gasp of the OSI protocol "suite". (At least I hope so). UTF-8, UTF-16, binary blobs. And topped off with something that is sort of like DNS name compression. Things like this are why I got into this business. At least, that is what I am telling myself to make my meaningless existence a little less horrible.

Anyway, back in receiveLdapPingReply, I called created an instance of SearchResultEntryMessage to parse the byte array I receive from the Active Directory server. SearchResultEntryMessage is the inverse of SearchRequestMessage, a message identifier followed by a SearchResultEntry component.

public class SearchResultEntryMessage extends ASN1Sequence
{
  private int messageId;
  private SearchResultEntry results;
  
  public SearchResultEntryMessage(InputStream is) throws IOException
  {
    ASN1Identifier id = new ASN1Identifier(is);
    if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || !id.getConstructed() || id.getTag() != ASN1Sequence.TAG)
    {
      throw new IOException("Illegal LDAP Message: expecting LDAPMessage");
    }
    this.setIdentifier(id);
    ASN1Length l = new ASN1Length(is);

    id = new ASN1Identifier(is);
    if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || id.getConstructed() || id.getTag() != ASN1Integer.TAG)
    {
      throw new IOException("Illegal LDAP Message: expecting Integer");
    }
    l = new ASN1Length(is);
    LBERDecoder decoder = new LBERDecoder();
    ASN1Integer messageId = new ASN1Integer(decoder, is, l.getLength());
    this.add(messageId);
    this.messageId = messageId.intValue();

    id = new ASN1Identifier(is);
    if (id.getASN1Class() != ASN1Identifier.APPLICATION || !id.getConstructed() || id.getTag() != SearchResultEntry.TAG)
    {
      throw new IOException("Illegal LDAP Message: expecting LDAP SearchResultEntry");
    }
    l = new ASN1Length(is);
    SearchResultEntry searchResultEntry = new SearchResultEntry(decoder, is, l.getLength());
    this.add(searchResultEntry);
    this.results = searchResultEntry;
  }
  
  public int getMessageId()  { return messageId; }

  public SearchResultEntry getResults()  { return results; }
 
  public String getObjectName()  { return getResults().getObjectName(); }

  public Map<String,Set<byte[]>> getAttributes()  { return getResults().getAttributes(); }
  
  public byte[] getNetlogon()
  {
    Iterator<byte[]> it = getAttributes().get("netlogon").iterator();
    while (it.hasNext())
    {
      return it.next();
    }
    return null;
  }
}

Parsing BER-encoded ASN.1 is somewhat more complex than generating it, at least using this interface. Specifically, the method needs to read and verify the tag in the byte stream, then the length, then parse the contents. And since ASN.1 is not entirely self-descriptive, I need a recursive descent parser rather than a few straight, flat readers. The other half of this class has a number of accessors for retrieving the data from the message once it has been parsed.

The SearchResultEntry class is mostly similar, although somewhat simpler. The objectName should always be an empty string, since it will refer to the root object of the LDAP tree. There is one tricky bit to this, specifically the PartialAttributeList.

public class SearchResultEntry extends ASN1Sequence
{
  public static final int TAG = 4;

  private String objectName;
  private PartialAttributeList attributeList;

  public SearchResultEntry(ASN1Decoder decoder, InputStream is, int length) throws IOException
  {
    this.setIdentifier(new ASN1Identifier(ASN1Identifier.APPLICATION, true, TAG));

    ASN1Identifier id = new ASN1Identifier(is);
    if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || id.getConstructed() || id.getTag() != ASN1OctetString.TAG)
    {
      throw new IOException("Illegal LDAP Message: expecting OCTET STRING (objectName)");
    }
    ASN1Length l = new ASN1Length(is);
    ASN1OctetString objectName = new ASN1OctetString(decoder, is, l.getLength());
    this.add(objectName);
    this.objectName = objectName.stringValue();

    this.attributeList = new PartialAttributeList(((ASN1Sequence) decoder.decode(is)));
  }
  
  public String getObjectName()  { return objectName; }

  public Map<String,Set<byte[]>> getAttributes()  { return attributeList.getAttributes(); }
}

The SearchResultEntry contains a PartialAttributeList for the actual results of the search, the attributes associated with the object that is found. Fortunately for me, being lazy, ASN.1 BER is almost self-descriptive; once I reached the level of the PartialAttributeList, the remaining components are normal, universal ASN.1 elements. So, back up in the SearchResultEntry, I simply called the decoder to read those elements, passed them into this classes' constructor, and use this class to provide convenient access to the attributes in the result.

public class PartialAttributeList extends ASN1Sequence
{
  private Map<String, Set<byte[]>> attributes = new TreeMap<String, Set<byte[]>>();

  public PartialAttributeList(ASN1Sequence sequence)
  {
    // Compress the sequence into this object
    this.setIdentifier(sequence.getIdentifier());
    for (int i = 0; i < sequence.size(); ++i)
    {
      this.add(sequence.get(i));
    }

    // Pick out the useful values
    for (int i = 0; i < sequence.size(); ++i)
    {
      ASN1Sequence attributeDescription = (ASN1Sequence) sequence.get(i);
      String attribute = ((ASN1OctetString) attributeDescription.get(0)).stringValue();
      ASN1Set valSet = (ASN1Set) attributeDescription.get(1);
      Set<byte[]> values = new TreeSet<byte[]>();
      for (int j = 0; j < valSet.size(); ++j)
      {
        ASN1OctetString string = (ASN1OctetString) valSet.get(j);
        values.add(string.byteValue());
      }
      attributes.put(attribute, values);
    }
  }

  public Map<String,Set<byte[]>> getAttributes()  { return attributes; }
}

The final step is to unpack the NetLogon result from the server, which contains loads of information, some of which might be interesting to someone who knows much more about Microsoft than I do. This creates an instance of a subclass of NetLogon, since the formats of the various options are different.

public abstract class NetLogon
{
  public static final int LOGON_PRIMARY_QUERY = 7;
  public static final int LOGON_PRIMARY_RESPONSE = 12;
  public static final int LOGON_SAM_LOGON_REQUEST = 18;
  public static final int LOGON_SAM_LOGON_RESPONSE = 19;
  public static final int LOGON_SAM_PAUSE_RESPONSE = 20;
  public static final int LOGON_SAM_USER_UNKNOWN = 21;
  public static final int LOGON_SAM_LOGON_RESPONSE_EX = 23;
  public static final int LOGON_SAM_PAUSE_RESPONSE_EX = 24;
  public static final int LOGON_SAM_USER_UNKNOWN_EX = 25;

  protected ByteBuffer buffer;
  protected int opcode;
  
  public static NetLogon unpack(byte[] bytes)
  {
    ByteBuffer buffer = ByteBuffer.wrap(bytes);
    buffer.order(ByteOrder.LITTLE_ENDIAN);
    int opcode = buffer.getShort();
    switch (opcode)
    {
    case LOGON_SAM_LOGON_RESPONSE:
      return new SamLogonResponse(opcode, buffer);
    case LOGON_SAM_PAUSE_RESPONSE:
      return new SamLogonResponse(opcode, buffer);
    case LOGON_PRIMARY_RESPONSE:
      return new PrimaryResponse(opcode, buffer);
    case LOGON_SAM_LOGON_RESPONSE_EX:
      return new SamLogonResponseEx(opcode, buffer, false, false);
    case LOGON_SAM_PAUSE_RESPONSE_EX:
      return new SamLogonResponseEx(opcode, buffer, false, false);
    default:
      throw new IllegalArgumentException("cannot unpack NetLogon structure");
    }
  }
  
  public int getOpcode() { return opcode; }
  
  protected NetLogon(ByteBuffer buffer)  { this.buffer = buffer; }

This class contains a number of useful functions needed to read the fields from the NetLogon structure. The first such method is parseIpAddress, which reads a four-byte, network-byte-order IP address from the current location in the ByteBuffer. It accepts a host name that should go with the address, which ideally would be parsed earlier. Note that since the address is in network-byte-order, which is big-endian (i.e. most significant byte first), and the byte stream is being read in Microsoft's little-endian order (i.e. least significant byte first), this method reverses the addresses' bytes.

  protected InetAddress parseIpAddress(String dnsHostName)
  {
    byte[] addressBuffer = new byte[4];
    for (int i = 0; i < 4; ++i)
    {
      addressBuffer[4-i-1] = buffer.get();
    }
    try
    {
      return InetAddress.getByAddress(dnsHostName, addressBuffer);
    }
    catch (UnknownHostException ex)
    {
      // cannot happen
      throw new IllegalStateException("cannot retrieve IP address from buffer");
    }
  }

Probably the second-simplest parsing method is one which reads a two-byte, zero-terminated UTF-16 character string from the buffer.

  protected String parseString()
  {
    StringBuilder sb = new StringBuilder();
    char c = buffer.getChar();
    while (c != 0)
    {
      sb.append(c);
      c = buffer.getChar();
    }
    return sb.toString();
  }

Perhaps the other, second-simplest method is one to parse a zero-terminated, UTF-8 character string from the buffer.

  protected String parseAsciiString()
  {
    StringBuilder sb = new StringBuilder();
    byte c = buffer.get();
    while (c != 0)
    {
      sb.append(c);
      c = buffer.get();
    }
    return sb.toString();
  }

But the simplest is one which just reads an array of bytes.

  protected byte[] parseByteArray(int length)
  {
    byte[] bytes = new byte[length];
    buffer.get(bytes, 0, length);
    return bytes;
  }

On the other hand, the hands-down, most complicated one parses a DNS domain name from the current location in the buffer. The domain name uses "pseudo-DNS compression", where each segment of the DNS name is encoded as a length byte followed by ASCII bytes. If the length byte has the 0xC0 bits set, the next byte represents a location in the buffer where the next segment of the DNS name is found. The process terminates when a zero-length segment is found.

An example:

• foo.example.com: "3foo7example3com0".


• bar.example.com immediately following the previous example, which starts the buffer: "3bar*4". The "*" represents the 0xC0 byte; the 4 is the location of the "7exam....".

Note: Real DNS compression encodes the location in the lower bits of the 0xC0 byte, if I remember correctly.

Microsoft's documentation does not even try to describe this scheme as well as I have; it simply presents an overly complex block of pseudo-code that parses all of the DNS-compressed host names at once.

  protected String parseDomainName()
  {
    StringBuilder sb = new StringBuilder();
    int current = buffer.position();    // "Current" location in the buffer
    int highwater = current;            // Highest current location seen; used to update buffer.position
    boolean firstLabel = true;
    
    while (true)
    {
      int labelSize = buffer.get(current);
      current++; highwater = (current > highwater) ? current : highwater;
      if (labelSize == 0)
      {
        // end of domain name
        break;
      }
      else if ((labelSize & 0xC0) != 0)
      {
        // redirect to another location in the buffer
        int next = buffer.get(current);
        // Maintain highwater as the highest location read in the buffer
        current++; highwater = (current > highwater) ? current : highwater;
        // ...even in the case where we would be redirecting forward
        current = next; highwater = (current > highwater) ? current : highwater;
      }
      else
      {
        if (!firstLabel)
        {
          sb.append('.');
        }
        firstLabel = false;
        for (int i = 0; i < labelSize; ++i)
        {
          int ch = buffer.get(current);
          current++; highwater = (current > highwater) ? current : highwater;
          sb.append((char) ch);
        }
      }
    }
    
    buffer.position(highwater);
    return sb.toString();
  }

}

For the sake of my own remaining sanity, I am going to elide (1) the versions of the NetLogon structure that I have not tested, and (2) most of the rather tedious elements of the remaining class, the one which implements the NETLOGON_SAM_LOGON_RESPONSE structure. Here is the constructor, which uses the methods in the superclass to parse the fields.

  public SamLogonResponse(int opcode, ByteBuffer buffer)
  {
    super(buffer);
    this.opcode = opcode;
    logonServer = parseString();
    userName = parseString();
    domainName = parseString();
    domainGuids = parseByteArray(16);
    nullGuids = parseByteArray(16);
    dnsForestName = parseDomainName();
    dnsDomainName = parseDomainName();
    dnsHostName = parseDomainName();
    address = parseIpAddress(dnsHostName);
    flags = buffer.getInt();
    ntVersion = buffer.getInt();
  }

So, how do you authenticate, once you have located the proper server? How does four lines of code strike you?

      LDAPConnection connection = new LDAPConnection( new LDAPJSSEStartTLSFactory() );
      connection.connect(records.get(messageId).getTarget().toString(), records.get(messageId).getPort());
      connection.startTLS();
      connection.bind(LDAPConnection.LDAP_V3, username+"@"+domain, password.getBytes());

There are a few important details in this code, though. First, to avoid sending the username and password over the network in the clear, the connection uses the StartTLS option before attempting the bind. (The other alternative would be an SSL connection, which would have to run on another port. TLS and SSL are closely related, but TLS can operate as an option inside another protocol and can run on top of the existing connection.) Second, and more importantly, is that in our environment the startTLS call requires a certificate from the Active Directory server to be validated, based on a certificate authority key that the program should already have (see the javax.net.ssl.trustStore property). Otherwise, the Active Directory server we are connected to could be a evil nogoodnik's simple way to collect user capabilites and do evil things with them, known in the security trade as a man-in-the-middle attack. (The fake AD server records user's information and passes the connection on to a real AD server for the domain.) Third, the user identifier for the bind is the user's account name concatenated with "@" and the fully-qualified domain name. This is unlike normal LDAP authentication, which requires first searching the LDAP tree for the user's distinguished name to use as the bind argument.

Now, what was I on about at the start of this post? Oh, yes, this process contains the entire Internet. The final piece that brought me to that conclusion was the byte ordering of the SearchResultEntryMessage. The message is encoded using the Basic Encoding Rules of ASN.1, which seems to specify that multi-byte integer values (such as lengths longer than one byte would hold) would be in network byte order, which is big-endian. The NetLogon structure, as I mentioned above, is (mostly) in little endian order. I can feel the flames of that particular war from here.

(Nogoodnik? And, I'm never going to post code using Java Generics again. < indeed.)