Authenticating against an Active Directory Server, pt. 5

Posted on September 26, 2009 by Tommy McGuire
Labels: authentication, protocols, ldap, active directory, java

With this post, I am getting to the actual punch line of the series, authenticating a username and password to a dynamically identified Active Directory server. Further, I just made a realization: this last post contains the entire Internet. The whole ball of wax is here, except perhaps for spam. A protocol defined to run on TCP but transported instead by UDP. ASN.1 and LDAP, the last gasp of the OSI protocol "suite". (At least I hope so). UTF-8, UTF-16, binary blobs. And topped off with something that is sort of like DNS name compression. Things like this are why I got into this business. At least, that is what I am telling myself to make my meaningless existence a little less horrible.

Anyway, back in receiveLdapPingReply, I called created an instance of SearchResultEntryMessage to parse the byte array I receive from the Active Directory server. SearchResultEntryMessage is the inverse of SearchRequestMessage, a message identifier followed by a SearchResultEntry component.

public class SearchResultEntryMessage extends ASN1Sequence
{
private int messageId;
private SearchResultEntry results;

public SearchResultEntryMessage(InputStream is) throws IOException
{
ASN1Identifier id = new ASN1Identifier(is);
if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || !id.getConstructed() || id.getTag() != ASN1Sequence.TAG)
{
throw new IOException("Illegal LDAP Message: expecting LDAPMessage");
}
this.setIdentifier(id);
ASN1Length l = new ASN1Length(is);

id = new ASN1Identifier(is);
if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || id.getConstructed() || id.getTag() != ASN1Integer.TAG)
{
throw new IOException("Illegal LDAP Message: expecting Integer");
}
l = new ASN1Length(is);
LBERDecoder decoder = new LBERDecoder();
ASN1Integer messageId = new ASN1Integer(decoder, is, l.getLength());
this.add(messageId);
this.messageId = messageId.intValue();

id = new ASN1Identifier(is);
if (id.getASN1Class() != ASN1Identifier.APPLICATION || !id.getConstructed() || id.getTag() != SearchResultEntry.TAG)
{
throw new IOException("Illegal LDAP Message: expecting LDAP SearchResultEntry");
}
l = new ASN1Length(is);
SearchResultEntry searchResultEntry = new SearchResultEntry(decoder, is, l.getLength());
this.add(searchResultEntry);
this.results = searchResultEntry;
}

public int getMessageId() { return messageId; }

public SearchResultEntry getResults() { return results; }

public String getObjectName() { return getResults().getObjectName(); }

public Map<String,Set<byte[]>> getAttributes() { return getResults().getAttributes(); }

public byte[] getNetlogon()
{
Iterator<byte[]> it = getAttributes().get("netlogon").iterator();
while (it.hasNext())
{
return it.next();
}
return null;
}
}

Parsing BER-encoded ASN.1 is somewhat more complex than generating it, at least using this interface. Specifically, the method needs to read and verify the tag in the byte stream, then the length, then parse the contents. And since ASN.1 is not entirely self-descriptive, I need a recursive descent parser rather than a few straight, flat readers. The other half of this class has a number of accessors for retrieving the data from the message once it has been parsed.

The SearchResultEntry class is mostly similar, although somewhat simpler. The objectName should always be an empty string, since it will refer to the root object of the LDAP tree. There is one tricky bit to this, specifically the PartialAttributeList.

public class SearchResultEntry extends ASN1Sequence
{
public static final int TAG = 4;

private String objectName;
private PartialAttributeList attributeList;

public SearchResultEntry(ASN1Decoder decoder, InputStream is, int length) throws IOException
{
this.setIdentifier(new ASN1Identifier(ASN1Identifier.APPLICATION, true, TAG));

ASN1Identifier id = new ASN1Identifier(is);
if (id.getASN1Class() != ASN1Identifier.UNIVERSAL || id.getConstructed() || id.getTag() != ASN1OctetString.TAG)
{
throw new IOException("Illegal LDAP Message: expecting OCTET STRING (objectName)");
}
ASN1Length l = new ASN1Length(is);
ASN1OctetString objectName = new ASN1OctetString(decoder, is, l.getLength());
this.add(objectName);
this.objectName = objectName.stringValue();

this.attributeList = new PartialAttributeList(((ASN1Sequence) decoder.decode(is)));
}

public String getObjectName() { return objectName; }

public Map<String,Set<byte[]>> getAttributes() { return attributeList.getAttributes(); }
}


The SearchResultEntry contains a PartialAttributeList for the actual results of the search, the attributes associated with the object that is found. Fortunately for me, being lazy, ASN.1 BER is almost self-descriptive; once I reached the level of the PartialAttributeList, the remaining components are normal, universal ASN.1 elements. So, back up in the SearchResultEntry, I simply called the decoder to read those elements, passed them into this classes' constructor, and use this class to provide convenient access to the attributes in the result.

public class PartialAttributeList extends ASN1Sequence
{
private Map<String, Set<byte[]>> attributes = new TreeMap<String, Set<byte[]>>();

public PartialAttributeList(ASN1Sequence sequence)
{
// Compress the sequence into this object
this.setIdentifier(sequence.getIdentifier());
for (int i = 0; i < sequence.size(); ++i)
{
this.add(sequence.get(i));
}

// Pick out the useful values
for (int i = 0; i < sequence.size(); ++i)
{
ASN1Sequence attributeDescription = (ASN1Sequence) sequence.get(i);
String attribute = ((ASN1OctetString) attributeDescription.get(0)).stringValue();
ASN1Set valSet = (ASN1Set) attributeDescription.get(1);
Set<byte[]> values = new TreeSet<byte[]>();
for (int j = 0; j < valSet.size(); ++j)
{
ASN1OctetString string = (ASN1OctetString) valSet.get(j);
values.add(string.byteValue());
}
attributes.put(attribute, values);
}
}

public Map<String,Set<byte[]>> getAttributes() { return attributes; }
}


The final step is to unpack the NetLogon result from the server, which contains loads of information, some of which might be interesting to someone who knows much more about Microsoft than I do. This creates an instance of a subclass of NetLogon, since the formats of the various options are different.

public abstract class NetLogon
{
public static final int LOGON_PRIMARY_QUERY = 7;
public static final int LOGON_PRIMARY_RESPONSE = 12;
public static final int LOGON_SAM_LOGON_REQUEST = 18;
public static final int LOGON_SAM_LOGON_RESPONSE = 19;
public static final int LOGON_SAM_PAUSE_RESPONSE = 20;
public static final int LOGON_SAM_USER_UNKNOWN = 21;
public static final int LOGON_SAM_LOGON_RESPONSE_EX = 23;
public static final int LOGON_SAM_PAUSE_RESPONSE_EX = 24;
public static final int LOGON_SAM_USER_UNKNOWN_EX = 25;

protected ByteBuffer buffer;
protected int opcode;

public static NetLogon unpack(byte[] bytes)
{
ByteBuffer buffer = ByteBuffer.wrap(bytes);
buffer.order(ByteOrder.LITTLE_ENDIAN);
int opcode = buffer.getShort();
switch (opcode)
{
case LOGON_SAM_LOGON_RESPONSE:
return new SamLogonResponse(opcode, buffer);
case LOGON_SAM_PAUSE_RESPONSE:
return new SamLogonResponse(opcode, buffer);
case LOGON_PRIMARY_RESPONSE:
return new PrimaryResponse(opcode, buffer);
case LOGON_SAM_LOGON_RESPONSE_EX:
return new SamLogonResponseEx(opcode, buffer, false, false);
case LOGON_SAM_PAUSE_RESPONSE_EX:
return new SamLogonResponseEx(opcode, buffer, false, false);
default:
throw new IllegalArgumentException("cannot unpack NetLogon structure");
}
}

public int getOpcode() { return opcode; }

protected NetLogon(ByteBuffer buffer) { this.buffer = buffer; }

This class contains a number of useful functions needed to read the fields from the NetLogon structure. The first such method is parseIpAddress, which reads a four-byte, network-byte-order IP address from the current location in the ByteBuffer. It accepts a host name that should go with the address, which ideally would be parsed earlier. Note that since the address is in network-byte-order, which is big-endian (i.e. most significant byte first), and the byte stream is being read in Microsoft's little-endian order (i.e. least significant byte first), this method reverses the addresses' bytes.

protected InetAddress parseIpAddress(String dnsHostName)
{
byte[] addressBuffer = new byte[4];
for (int i = 0; i < 4; ++i)
{
addressBuffer[4-i-1] = buffer.get();
}
try
{
return InetAddress.getByAddress(dnsHostName, addressBuffer);
}
catch (UnknownHostException ex)
{
// cannot happen
throw new IllegalStateException("cannot retrieve IP address from buffer");
}
}

Probably the second-simplest parsing method is one which reads a two-byte, zero-terminated UTF-16 character string from the buffer.

protected String parseString()
{
StringBuilder sb = new StringBuilder();
char c = buffer.getChar();
while (c != 0)
{
sb.append(c);
c = buffer.getChar();
}
return sb.toString();
}

Perhaps the other, second-simplest method is one to parse a zero-terminated, UTF-8 character string from the buffer.

protected String parseAsciiString()
{
StringBuilder sb = new StringBuilder();
byte c = buffer.get();
while (c != 0)
{
sb.append(c);
c = buffer.get();
}
return sb.toString();
}

But the simplest is one which just reads an array of bytes.

protected byte[] parseByteArray(int length)
{
byte[] bytes = new byte[length];
buffer.get(bytes, 0, length);
return bytes;
}

On the other hand, the hands-down, most complicated one parses a DNS domain name from the current location in the buffer. The domain name uses "pseudo-DNS compression", where each segment of the DNS name is encoded as a length byte followed by ASCII bytes. If the length byte has the 0xC0 bits set, the next byte represents a location in the buffer where the next segment of the DNS name is found. The process terminates when a zero-length segment is found.

An example:

Note: Real DNS compression encodes the location in the lower bits of the 0xC0 byte, if I remember correctly.

Microsoft's documentation does not even try to describe this scheme as well as I have; it simply presents an overly complex block of pseudo-code that parses all of the DNS-compressed host names at once.

protected String parseDomainName()
{
StringBuilder sb = new StringBuilder();
int current = buffer.position(); // "Current" location in the buffer
int highwater = current; // Highest current location seen; used to update buffer.position
boolean firstLabel = true;

while (true)
{
int labelSize = buffer.get(current);
current++; highwater = (current > highwater) ? current : highwater;
if (labelSize == 0)
{
// end of domain name
break;
}
else if ((labelSize & 0xC0) != 0)
{
// redirect to another location in the buffer
int next = buffer.get(current);
// Maintain highwater as the highest location read in the buffer
current++; highwater = (current > highwater) ? current : highwater;
// ...even in the case where we would be redirecting forward
current = next; highwater = (current > highwater) ? current : highwater;
}
else
{
if (!firstLabel)
{
sb.append('.');
}
firstLabel = false;
for (int i = 0; i < labelSize; ++i)
{
int ch = buffer.get(current);
current++; highwater = (current > highwater) ? current : highwater;
sb.append((char) ch);
}
}
}

buffer.position(highwater);
return sb.toString();
}

}

For the sake of my own remaining sanity, I am going to elide (1) the versions of the NetLogon structure that I have not tested, and (2) most of the rather tedious elements of the remaining class, the one which implements the NETLOGON_SAM_LOGON_RESPONSE structure. Here is the constructor, which uses the methods in the superclass to parse the fields.

public SamLogonResponse(int opcode, ByteBuffer buffer)
{
super(buffer);
this.opcode = opcode;
logonServer = parseString();
userName = parseString();
domainName = parseString();
domainGuids = parseByteArray(16);
nullGuids = parseByteArray(16);
dnsForestName = parseDomainName();
dnsDomainName = parseDomainName();
dnsHostName = parseDomainName();
address = parseIpAddress(dnsHostName);
flags = buffer.getInt();
ntVersion = buffer.getInt();
}

So, how do you authenticate, once you have located the proper server? How does four lines of code strike you?

LDAPConnection connection = new LDAPConnection( new LDAPJSSEStartTLSFactory() );
connection.connect(records.get(messageId).getTarget().toString(), records.get(messageId).getPort());
connection.startTLS();
connection.bind(LDAPConnection.LDAP_V3, username+"@"+domain, password.getBytes());

There are a few important details in this code, though. First, to avoid sending the username and password over the network in the clear, the connection uses the StartTLS option before attempting the bind. (The other alternative would be an SSL connection, which would have to run on another port. TLS and SSL are closely related, but TLS can operate as an option inside another protocol and can run on top of the existing connection.) Second, and more importantly, is that in our environment the startTLS call requires a certificate from the Active Directory server to be validated, based on a certificate authority key that the program should already have (see the javax.net.ssl.trustStore property). Otherwise, the Active Directory server we are connected to could be a evil nogoodnik's simple way to collect user capabilites and do evil things with them, known in the security trade as a man-in-the-middle attack. (The fake AD server records user's information and passes the connection on to a real AD server for the domain.) Third, the user identifier for the bind is the user's account name concatenated with "@" and the fully-qualified domain name. This is unlike normal LDAP authentication, which requires first searching the LDAP tree for the user's distinguished name to use as the bind argument.

Now, what was I on about at the start of this post? Oh, yes, this process contains the entire Internet. The final piece that brought me to that conclusion was the byte ordering of the SearchResultEntryMessage. The message is encoded using the Basic Encoding Rules of ASN.1, which seems to specify that multi-byte integer values (such as lengths longer than one byte would hold) would be in network byte order, which is big-endian. The NetLogon structure, as I mentioned above, is (mostly) in little endian order. I can feel the flames of that particular war from here.

(Nogoodnik? And, I'm never going to post code using Java Generics again. &lt; indeed.)

Comments



Just a question about the parts of the code (NetLogon subclasses) you haven't posted - would you mind doing so? (all threats to sanity notwithstanding) - unless they are different from the SamLogonResponse class in having uninherited methods, perhaps just the constructors?



This has been an extremely illuminating post - one of few where I've actually been able to understand what's going on. Thanks!

David Sills
2013-12-12

Would you mind posting the other NetLogon subclasses (at least constructors, assuming they have no uninherited methods)?




This has been one of the few posts of this type where I've been able to understand what's going on. Thanks!

David Sills
2013-12-12
active directory applied formal logic ashurbanipal authentication books c c++ comics conference continuations coq data structure digital humanities Dijkstra eclipse virgo electronics emacs goodreads haskell http java job Knuth ldap link linux lisp math naming nimrod notation OpenAM osgi parsing pony programming language protocols python quote R random REST ruby rust SAML scala scheme shell software development system administration theory tip toy problems unix vmware yeti
Member of The Internet Defense League
Site proudly generated by Hakyll.