Why Failure Is A Part Of The Road To Success (Calculation Engine Case)

Today I will tell you about the experience I gained developing one of our projects. An important conclusion I made afterwards is that it’s never too late to start it all over again. Do not be afraid to make mistakes, every mistake can turn into a hint and the first step to something new.

We became part of the Lean Case project in early 2015. The main purpose of this project is to enable business owners to control their business. Moreover, the business area does not matter here. Lean Case helps you control your personal income, earnings from each employee, plan new goals, and etc.

This project is based on mathematical and economic models, with the help of which the control is performed. There are plenty of models, the customer can come up with new models every day. Therefore, the main idea of the Lean Case was in these models’ program implementation.

Sure enough, we did not create these models, because in order to create them it was necessary to have profound mathematical and economic experience under belt. It was our customer who was responsible for this, he performed as these models moderator. For their creation he used Excel, namely this program was a connecting link between us and the client. It was not important for us to think much about what the model estimates. Our task was to program the model and output the desired results on its basis.

Here is an example of one of the models:

Mathematic model sample
This model has input and formulated data. Now, for a moment, imagine what an “exciting” job is – to explore Excel and program these algorithms.

Wanna get out work early

Each model is a separate module that can be used in other models. In terms of programming, the model is a separate class with a large number of methods and properties, and this class must necessarily comply with SOLID principles.

At the beginning of the project development, the customer had 10 such models. It seems as simple as it is, all you need is to write 10 classes and that’s it. Having programmed the last 10th model, we began to prepare the project for release in the beta version. The beta version release day came faster than we expected.

It had been only a month since the project was started in beta version, but it already began to interest the potential audience. In particular, several banks and other financial institutions began to offer their models!

Good job!

The development of models continued, we received more and more new excel-files describing these models. The volume of work increased and it became more complicated. It took 3-4 days to program one Java model. Development continued. Each subsequent model worsened the atmosphere and team’s mood. We needed to change something, because we just did not see the end of this process 🙁

Just to much responcibility

The most important task for us was to get the message to the client. Therefore, on November 4, 2016 there was a call with the customer.

The call began with joyous news from him:
“Guys, that is great success – Steve Blank got interested in the project,” the godfather of the Silicon Valley “- wants to try the Lean Case as an educational tool for students at one of the best universities in the world – Stanford!”

But Steve’s main requirement was that the service should work with the new models that Blank had invented. Deadline from Steve – 3 months. The call ended on good terms, there wasn’t said a word about the bad attitude and atmosphere in the team. There was no choice left anyway, so we had to continue to move with the same old approach that we did not like.

The next day, a local meeting with the team took place.

Team took place

Having gathered at a round table, we began to analyze the project. It took a lot of time, but nevertheless, we came to a common opinion. After analyzing the time we spent on the development of the project, we realized that the factor that took most of the time was the study and programming of Excel-models.

The decision we took was obvious – to abandon Excel-models programming and their research. This was just the moment when we realized that we needed another approach there.

We needed to find an alternative to Excel that would be compatible with JAVA. But, unfortunately, there was nothing like that at that moment. And there came an idea from our Team-leader: “It would be great to get models from the client and financial institutions in a format that would be easily supported by JAVA.”

We need to do good

It was then when we came up with the idea of developing a calculating engine for supporting models that will meet our demands. The main key should be that following the engine algorithms, anyone would be able to develop models, not only financial institutions representatives and the customer, but also anyone who wants to.

Having analyzed all possible future scenarios, we abandoned the Excel-models programming approach. We understood that we had only 3 months, so it was decided to leave the version with current models and write a new one with Blanc’s models. Subconsciously, we understood that developing something better than Excel having only 3 months would be quite a hard task.

The next stage were tense negotiations with the client. The most difficult task was to make him understand that the project, which had been being developed for about a year, should cease to exist.

For quite a long time I have been thoroughly preparing to this conversation; different thoughts were running through my head, my colleagues tried to give me some advice; There were many jokes as well as serious thoughts, and only after an hour of discussions, having plucked up the courage, with the support of my colleagues, I opened my laptop and authorized for Skype.

What immediately struck me was a customer’s green color status, which was different from a bunch of other my contacts with yellow ones. For a moment, I imagined his facial expression when he was told that the project in which he invested a fortune – needs to be closed.

What??

When I stopped speaking there was a minute silence in the handset… Then there was a conversation about a lot of money invested in the project. The culmination moment was the story and live-demo of the new approach for models. The customer appreciated what he saw, but he realized that the development of such a mechanism requires a lot of time. The question of time we took upon ourselves. It was decided to expand the team and work without weekends!

In addition, it was agreed to leave the version with current models and write a new one with a new approach based on Blanc models. And then, gradually, implement the migration from the old version to the new one.

Discussing new approach

Discussing new approach

As I wrote earlier, we needed a new flexible model developing approach, alternative to Excel, so that the model could then be written by anyone following our algorithms. It was necessary to develop a calculation engine.

Relying on the experience, many cases were obvious. We had to foresee all cases for the basic work of the engine and provide easy and flexible code support.

The new approach was to develop a mechanism (calculating machine), which would accept the model at the input and immediately give the desired result. The source of understanding was chosen the JSON model representation. It was also important to make the model structure as simple as possible so that later they could be created by anyone. The team was expanded.

Many sleepless nights … and a basic version of the calculative (Lean-Case) engine has been developed!

The development of the base version took about two months. At that time, for Blank models implementation we had less than a month. Since there were forseen many cases, the implementation of models into our engine language was performed quite quickly. Out of the 6 models that Blank provided, 5 were adapted to our engine. The second version was released. Students’ had their training. Lean Case has gained.

Worldwide popularity!

At present, the project with a new approach has been under development for more than a year. Now we are not responsible for the study of boring Excel and mathematical algorithms programming. For this is responsible our engine with documentation, on the basis of which, the model can be written by anyone. Now our main task is to teach our engine to count new, unknown models for it.

So far, the Lean Case project has already been represented at many world business presentations. Lean Case is now considered to be the main competitor of the companies depicted below.

Lean Case Ecosystem

Brief engine’s preview

Input model example:
Input model example
Link

Resolved model example:
Resolved model example
Link

I will not describe how exactly engine supports the model in detail; for this purpose one more article is needed, and a big one. In the nearest future it will be written by my colleague, the developer of the engine’s basic version.

So briefly. The input model must be in JSON format. The main attributes of the model are name type priority and fields. Fields are the model’s main elements. The model may contain an undefined number of fields. The field is certain data, metrics for output.
Fields can have the following specific types:

  • DATE
  • DRIVER_TYPE
  • NUMBER
  • LIST
  • FORMULA
  • PERCENT
  • TEXT

Fields are also divided into input fields and fields with result. An important role is played by the priority of the fields, because the results can be used in solving other fields. Engine’s main task is to solve the fields.

Fields are resolved one by one as a pipeline job. The fields’ solution consists of 7 stages:

  • Resolve arguments
  • Resolve custom variables
  • Resolve variables
  • Resolve variable from dependent model
  • Resolve function
  • (%function_name(?field_name?,*field_name*)%)

Markers in JSON:

  • (*field_name*)
  • (?field_name?)
  • ($field_name$)
  • (#$modelReference$.field_name#)
  • (%function_name(?field_name?,*field_name*)%)

The two steps below come into action when all the top 5 stages are passed. Groovy shell resolves the formula and issues a result in the form of a map:

  • Resolve formula
  • Resolve attribute

Input field example from the model:

Input field example from the model

Resolved field example:

Link

It’s truly said that sometimes a failure is a part of the road to success. And our experience is a great confirmation of that saying.

Andriy Ostapko
Andriy Ostapko
Java Software Developer