Alex Au (left) and Shazwi Suwandi, the software engineer

TWC2 vice-president Alex Au was the lead on the client (TWC2) side for the development of our second case-management system (“Camans v2”). We ask him about some of the key learnings in the course of this project.

TWC2 had a digital case management system for years. Why was there a need for a new version?

Indeed, we were very happy with the system built by final-year students of Singapore Management University which went live in March 2015. See New case management software comes on stream . It’s been robust and reliable all these years. That team of students won a prize for it in 2015, but I think the greater accolade is that it’s been seriously useful to us all this while.

However, the nature of TWC2’s work has gradually evolved over time, and there were new needs which slowly revealed themselves to be important, and yet were not fully supported by Camans v1. At some point, we felt that we needed a version 2.

What were some of these new features that were now needed?

There were several, but I’ll just name four as examples.

In 2015, we didn’t realise how frequently workers would be coming back to us with new problems under new employers. This “repeat custom”, as one might call it, wasn’t so evident in the early years of TWC2. As time passed, we saw more and more workers coming to us, years after their first cases, with new issues that they needed help with. While version 1 was very good at handling the here and now, it wasn’t as good in handling and displaying historical information. In drawing up our specifications for v2, we felt that unless we had a more sophisticated system for segregating the details of the old case from the current case, there was a risk of confusion among our users between one and the other.

There was also a need to incorporate personal data consent forms (“PDPA forms”) within the case management system. Version 1 did not have this feature, and for years we had paper forms manually signed by workers and filed separately. We wanted to get rid of paper and in 2019 we moved to digitally-signed consent forms. Once we did that, it became obvious that they should also be integrated with Camans.

We wanted a finer-grained handling of sensitive information. For instance, since 2015, we’ve had a few court cases where the judge imposed a gag order on the identity of the persons involved. With version 1, we had to carve these cases out of Camans and put the information on paper files under lock and key; but obviously it was not efficient to have most cases in the digital system, but some cases on paper. Version 2 gives us a more sophisticated way of restricting access to these cases to just a few caseworkers on a need-to-know basis.

Over the past few years we found ourselves dealing with large groups of workers from the same employer coming to us together. Normally, our case handling protocol would be to handle cases individually since no two workers’ cases are identical. However, in instances where they were really a batch of workers sharing a similar problem, the efficient way would be to handle them as a batch. Camans v1 wasn’t designed to accommodate this. We decided that v2 should give us the option of dealing with the workers sometimes as a batch, sometimes individually. That would require some complex features in v2.

What is really striking is that this is a custom-built system. Weren’t there off-the-shelf options?

In fact, at the beginning of our search for an upgraded case management system, we thought we needed to look no further than an off-the shelf system. However, it turned out that many of them were subscription-model systems, which made absolutely no sense for TWC2.

Subscription models often operate on the basis of a fee per user. This may seem affordable for small businesses with, say, 10 or 12 employees working fulltime, each spending eight hours a day using the system. But a charity organisation like TWC2 has, at any given time, as many as a hundred volunteers needing access to case data (at various levels of permission). Yet a typical volunteer only comes in perhaps two hours a fortnight. How is paying subscription for 100 – 200 accounts, each with low usage, ever going to make sense, especially when the annual fee per user can run into the hundreds of dollars?

We did find a few off-the shelf solutions that could be sold to us at a one-time cost. We spent two years trying to get one to work in the way we wanted. While it was supposed to be customisable to the client’s needs, in truth there were limits to what could be customised. In the one we were working with, some parts were hard-coded and there was no way we could modify them to our needs. We found ourselves having to make compromises to our feature requirements to fit the pre-built system – hardly a satisfactory way to proceed.

After two years of banging our heads over this, we realised there was no getting away from the fact that, for an organisation like TWC2 with such specific requirements, a custom-build was the best way forward even if, on the face of it, it sounded crazy. We ultimately realised that the work involved in customising an existing app – and even then, we had to scale back several features that we wanted – was probably going to be much more than building something from scratch.

Camans v2, as built by Shazwi Suwandi, incorporates every feature on our requirements list. We did not have to make a single compromise. How cool is that?

All our client discussions are logged and minuted in our case management system so that other caseworkers and volunteers can step in if needed to assist with a case.

Wouldn’t a custom build take much longer?

I don’t know. I guess it depends on the specifics of a project; I’m not in a position to draw general conclusions.

What I do know is that our attempts at customising an off-the-shelf system went on for two years, with ballooning frustration on our part, and was still a million miles from completion, whereas Shazwi took 15 months.

I think 15 months is great. Consider this: For Camans v1, we had four final-year students spending six to eight months building it – also as a custom build. That’s about 28 person-months. Shazwi did version 2 single-handedly in 15 months, and version 2 has many more features than version 1, such as direct and group chats, reports, PDPA integration and image manipulation.

At first, he estimated it would take six months. I kind of smiled at him but didn’t reveal what my thoughts were.

Of course, TWC2 had the benefit of being in a very comfortable position because version 1 was working fine. It did not have the new capabilities we would have liked, but it was a perfectly reliable workhorse and thus allowed us to be patient. So once again, we are very grateful to the SMU students who did the first version for us.

Custom-builds are far more prone to bugs. Were you concerned about that?

That’s true. It was perhaps the chief reason why we were reluctant to go that route until we had exhausted other possibilities. Off-the-shelf solutions would have been used by many other clients and one should expect few bugs to remain. That’s a good thing.

So, quite early on, as soon as we made the decision to go with Shazwi’s idea of a custom-build, we reached out to a retired IT lecturer in Malaysia to oversee the quality assurance side of the project. He roped in some IT students there to do testing as each section was coded. The lecturer designed the testing protocol and guided the young men accordingly. This went on for about six months during which they tested the Search and CRUD functions of every table at every role level. It was mechanical and numbing work, but they persevered.

That still left a lot of other functionalities such as reports, tagging users, tagging other workers, dropdowns and privileges management, for which, to be honest, we never quite had any systematic protocol. We tested and tested, but were we exhaustive? I don’t know.

We may well find issues with these functionalities in the next few weeks and months, but at least they don’t have much bearing on the integrity of the database itself, and thus are not likely to be system-critical.

Could you distil from this experience some key factors of success? Surely, extremely clear client specifications must be the starting point.

Very much so. I’ve done this kind of software implementation a few times in my career and experience has taught me the supreme importance of having a deep understanding of your own businesses processes and requirements. If you cannot communicate to your developers exactly what you want, you can kiss success goodbye.

And even then, people get things wrong. Too often, people describe business processes in terms of the “average” or “typical” scenario. The job of the analyst is to avoid this mental trap and instead think about all the possible edge cases. How is the system to behave if there are unusual inputs, far outside the normal? What are the subconscious assumptions we make about our cases which we must guard against, because inevitably, there will be a day when a case comes to us with characteristics or needs that are outliers; is the system robust and flexible enough to accommodate such cases? The “average” or “typical” case is easy to design for, but it’s the edge case that can break your system.

Designing a plane that can fly in clear weather is the easy part. Designing one that stays aloft in diabolical weather with no visibility and one engine flamed out is what you have to worry about.

Did TWC2 have a project committee to shepherd this project through? What’s the optimal size and composition of one?

Yes and no. There was a small group, each person with different expertise, whom I consulted with regularly, but there really wasn’t anything formal about it.

The software developer should have only one point of contact. The worst thing that can happen is if he has to work with different people each coming up with different requests. This means that the client organisation needs to have one trusted person who is (a) superbly familiar with the organisation’s processes and requirements and (b) be able to marshal ideas from others in the organisation and smooth out conflicting wish-lists – before they reach the developer.

The list of requirements has to be drawn up very, very early in the process and the client leader must have the authority to refuse late requests (but if he has been thorough in the early conceptual stages, there should be few late requests). It is very common for users to expect something to be built, then after a bit of testing, they ask for changes. Not only does this create a lot of reworking, changing one element after it has been built risks breaking another part of the system. So, whoever is drawing up the list of requirements has to do it purely in his imagination. He has to see the whole and final thing in his mind and how it works at the macro level and the micro level, without anything tangible to test-drive.

A volunteer (right) calls up QR codes on her screen which pull in personal data consent forms. Clients open the forms on their phones and after signing with their fingers, the completed forms go into Camans v2.

You also designed with data migration in mind, didn’t you?

Yes, we did. Legacy issues are often overlooked at the design stage, and that would be foolish. Especially with younger software engineers, their university projects seldom prioritise this side of things. It’s one thing to design a system that takes in data from the day of launch and does marvellous things with them, it’s another that can soak in tons of old data from a legacy system and still do marvellous things with them.

What it meant was that in designing Camans v2, we had to be highly conscious of how the data from Camans v1 had been structured. The data model of Camans v2 therefore had to correspond as far as possible to the data model of v1, and where it differed, we had to have a clear idea of how to convert and migrate such data into v2. From the outset, there had to be a migration plan, and not leave it as an afterthought.

It’s like I have a grand piano and I am building a new house. From the very beginning, I have to design a door wide enough to move the piano through. I shouldn’t have to hack down a wall after it’s been built.

Our brief to Shazwi included migration. I was highly conscious of the fact that this was not an area within his experience and at the beginning, I was a little anxious. In the end, he did it remarkably smoothly, writing all the necessary scripts to convert and move data from v1 to v2. It speaks volumes about his skill and quick learning that he managed to acquire what he needed to know within the space of only about a month. On migration day, a decade’s worth of data including document files were moved over in just minutes. It took just one attempt.

It sounds like you are delighted with how this project has worked out.

I think justifiably so. While the primary credit belongs to Shazwi Suwandi – he slogged at it nearly every evening and weekend for 15 months – lots of other people contributed magnificently. There were my colleagues in TWC2 who offered ideas about what new features would be good to have; others who guided Shazwi in setting up the system on the host server, plus security features, backups and firewalls, integration with external apps and tips about migration. There was the testing team in Malaysia and our caseworkers in the office here who were the proverbial guinea pigs in the trial phases.

Less obvious to the eye is the organisational culture here at TWC2. My colleagues trusted me without reservation to lead the project. There’s a respect for open channels of communication, a willingness to learn and adapt, and an almost childlike faith in technology.

I hope their faith has been validated. On Monday evening, an hour after Camans v2 went live, a team of volunteers were using it to update workers’ cases. Shazwi and I were on hand to take questions and provide help. We were called upon only rarely.

On Tuesday morning, I logged in from home to watch how the caseworkers in the office were responding to the system. To my pleasant surprise, they had logged in before me and were already using it extensively. No one was messaging me with questions. No one was asking for help.

As I watched the audit log silently reel in one new entry after another, I said to myself: This is a thing of beauty.