Processing of recursively defined XML objects with DataStage

Recursively or not…

Sometimes people get a bit over enthusiastic with define XML data objects using many XSDs including recursively defined objects in them. So basically defining objects with a definition of themselves… More or less…

So as this can be fun to the person who creates them, this can be a pain to others. So I got such a case…

So I got a bunch of XSDs in a ZIP file. Good thing to know is that the Schema Library Manager of DataStage or the XSD connector in Asset Manager both can handle ZIP files to import. As I am going for the DataStage path, I used the Schema Library Manager

And so created a library called Kadaster.

Here the steps to create a library if not know yet.

First thing to determine is what is really going wrong? In this case the customer was unable to map an object called “Perceel” which is part of the object “KadastraalObject”.

KadastraalObject itself was used in another object called object called “KadastraalObjectSnapshot”. A hint occurred here: I was not able to “dive” into KadastraalObject within a KadastraalObjectSnapshot.

In the XML dataset another hint occurred: In a KadastraalObjectSnapshot we go to a Perceel object “via” a KadastraalObject. This latest object is just an intermediate.

When looking into the XSD (text) of KadastraalObjectSnapshot there is a plain reference to KadastraalObject.

But when looking at the XSD (text) of KadastraalObject there is a (double) recursive definition for Perceel (and others). As stated by IBM support: Although allowed in XSD definitions, the hierarchical stage does not support them. That is the reason Perceel could not be mapped. So now what?

Let’s use chunking again!

Chunking does not care what is in the chuck: So in this case as we lost the “connection” to a KadastraalObject anyway, we chunk at KadastraalObjectSnapshot level.

With some string shifting, we loose the message parts we do not need. We only want the Perceel content.

Now feed the edited data to a second hierarchical stage that will parse it as “just” a Perceel, based upon its provided XSD.

Et voila!




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s