One of my latest blog entries have been about a security issue.
The sort of issues are as such fixable and often very easy fixes. We have both the knowledge and we have the tools to do so. As I have lined out on that earlier occasion, our organizations sometimes lack the process and QA it takes to make sure that we do not leave our sites vulnerable.
Anyway – working on the issue did however demonstrate some other problems, I had not anticipated.
Picking some modules from CPAN to help me out seemed like a good idea and the component in question was already relying heavily on CPAN modules.
When I first started out, I had separated out the component in question from our core software, the actual portal software. Since the component was actually just a plugin (we call these services), so this was quite easy. This would mean that I could work on it and deploy it without having to deploy a whole lot of unrelated software, which current state I was not totally updated on. Personally I would prefer to isolate and focus on the security issue at hand.
So I cooked up a new distribution, copied the stuff I needed (not much history lost there since we only had the initial revision), deleted it from the core trunk, no dual life for me no sir, and moved on…
I then started building a proper distribution. I threw in my scripted installation process based on a subclass of Module::Build, updated this to reflect that the service was a core server not living on the ordinary application hosts, but on the core hosts, this seemed like an easy fix, documenting it was the hardest part, but I was to become wiser.
I am of that opinion that we should not have first and second rate citizens in our system, a service is a service and should be agnostic to where it is invocated. There is however a big difference between the installations. Where our application servers have a designated special directory outside the control of the local Perl configuration, the core hosts install much of their stuff under the control of the local Perl configuration, this was a bit annoying, but certainly fixable, but it does mean that the two rates of services cannot rely on the same build tool as is. Even though they are both just services.
Another problem with the build tool was that it utilized newer features so I was not compatible with the installed Perl and I was unable to install a newer one, note on revisiting this was made.
Not having a proper build tool to assert my configuration and the sanity of what I was trying to do. I simply had to install the module, by overriding the existing one and trying it out.
First shot (my release 0.02) failed utterly. My use of XML::Simple was simply too fancy, so I had the same problem as I had experienced for Module::Build.
I rewrote the fix using a simpler strategy. This time with the 0.03 release candidate, it at least compiled.
So after a lot of hoop jumping it worked out after some more testing together with a colleague, we nailed the actual security problem.
So to get back to the intro in this blog entry.
We have the tools and we have the knowledge, but as this demonstrates I ended up implementing a less generic solution using a regular expression. Just running the data through some encoding schemes would keep me happier, since regular expressions are often culprits of bugs and the first solution was much more generic and would handle changes better.
So the hindrance for doing the right thing, was actually the platform. Time is slowly catching up with us and the technical debt put upon by using a platform, which has not been kept up to date is starting to interfere with our work and solutions. The platform primary goal is to bring leverage, but now it is putting constraints on us and the work we do.
It would be easy to update the Perl interpreter, but working under time constraints to get a security issue fixes is not the time and place. Updating the Perl interpreter would require extensive regression test, something I am not sure our platform is ready for.
The lesson learned must be that a platform should be maintained using baby-steps, this strategy has proven useful in other situations. This mean that you update in small gradual steps so you do not have to take huge giant leaps, when you finally get to a point where updating is unavoidable.
In the example above I simply added more to the technical debt than was necessary, but I was forced to. I did not make the giant leap, which would probably be a good idea, so the next issue would be easier to address using contemporary versions of the tools at hand.
Do not get me wrong I really like old platforms and in particular this platform, but things should be maintained, because if not they become a nuisance and get exchanged for something new and fancy, which might introduce new bugs, which had been addressed over time on the old platform.