Extending the regular expression

Having begun to form the expression we now have a complete scheme for selecting, and inserting commas for the first field in the line below (in blue);

• Search expression - ^(\d+(?=\x20))\x20
• Replace expression - \1,

What we need now, is a scheme to get at the part in green, and add a comma after it. Some time back we realised that we could use the the CLS field (in red) as a terminator to stop selection after the text field. All we need to do is to decide how to express the field in green, and the field in red.

Looking back at the whole dataset reveals that the majority of the green field on each of the 150 lines only contains digits word characters and spaces. With an eagle eye, one also realises that a couple of these fields contain hyphens and brackets. We can then, create the first part of our expression, [\d\w\s\-()]+. This is pretty straightforward. It's just a range that can match digits word characters and spaces, in addition brackets and hyphens. The range has the property of matching a single character from those described inside the square brackets. Obviously we have more than just the one so, like before, the plus indicates one or more.

What we need now is a way to terminate the greedy plus. We already decided that this would be the CLS field (in red). To terminate we use \x20\d{2}\x20. We already know about \x20, and that \d matches any digit. The only new thing, is the curly brackets. Like the plus these implement a repeat on the previous token, in this case digits. \x20\d{2}\x20, then matches a space followed by two digits, followed by a space.

At this point you may be wondering why we've included the additional trailing space. The reason is that the plus we're trying to terminate with it, is greedy. If we don't include the extra space the greedy plus, trying to match digits, words and spaces, will leap straight through the CLS field, because the terminator can match on the first numeric field "76" in black. Because of the trailing space, a failure to match occurs on the first "76" where the space collides with the decimal place period. The trailing space ensures that the terminator always terminates at the CLS field.

So then, to match the field in green we use [\d\w\s\-()]+(\x20\d{2}\x20). The problem with this is that it also matches the red CLS field. When we use the technique, previously described in detail, to move the characters outside, it doesn't matter that the terminator is longer than that which we wish to change. We are going to describe the terminator as a metatoken group, so even though it will perform the termination, it won't affect the changes we want to make in replacement. Using the technique, we get; ([\d\w\s\-()]+(?=\x20\d{2}\x20))\x20.

In this new expression, the body part (in green), is completely enclosed in the first group, and the trailing space is outside the group, just like before. If we were going to use this search expression on it's own, then we would use the same replacement expression as before. As it is, we haven't put any thought into where the expression is going to start matching. Because of this, the best thing to do is to concatenate it with the first expression, at the top of the page. What we actually want to do is have this expression follow on from that one.

Once concatenated we have the following search and replacement expressions;

• Search expression - ^(\d+(?=\x20))\x20([\d\w\s\-()]+(?=\x20\d{2}\x20))\x20
• Replace expression - \1,\2,

We can now run these expressions using a "Replace All" operation. The overall effect will be to add commas between the blue and green fields, and the green and red fields. This yields;