Loyalty Engine

---

The task here was to build an engine to handle Loyalty in a generic way. All parts of the engine had to be pluggable including the business rules and the loyalty algorithm itself. The process had to be scalable too on various different architectures.

The algorithm itself was technology agnostic - so the engine didnt care what tool was used to do that work whether it was R, SAS or in more simplistic cases Oracle PL/SQL

The business rules were extremely complex, and entirely metadata driven. This meant it was possible to change flow, algorithm precedence and more without changing a single line of code

The input process was also metadata driven and standardised. This meant it was possible in most cases to onboard new clients without changing any code.

The scale was challenging yet achievable - we had to process billions of message combinations.

• Client: A large listed company specialising in Loyalty and running the largest loyalty programme in the UK
• Date: 2014
• Service: ETL and analytics engine development

Close

Ingestion Engine

---

A large investment bank wanted an automated way to populate their data lake. The task was simply to:

• Read the metadata from JDBC databases, Excel, Lotus Notes and a myriad of flat files
• Present this to data owners so they can choose attributes and mask where necessary
• Load that data into AVRO files in Hadoop
• Maintain the HIVE tables according to the metadata
• Deal with schema change in a nice way

The challenge here was all of this had to be done without any code modification, so metadata was important.

By the end of the project the team had delivered a generic engine that could be used for any customer and had an extensive automated test suite. The test suite was necessary in this case because of the large number of people working on the project, and the different teams from different suppliers.

• Client: Large international investment bank
• Date: Dec 2014
• Service: ETL

Close

Greenfield Migration Project

---

Back in 2015 the government mandated that all its departments must move away from expensive proprietry software with ongoing maintenance charges. Additionally they also wanted to move away from huge recurring consulting contracts. There was a noted preference (albeit not mandated) to move to open source software.

The task here was to migrate about 200 data warehouses out of Oracle, DB2 and Sybase all into a huge datalake in Hadoop.

The architecture took an interesting approach. In some cases simply ingesting the source data was adequate, so an internal team handled that easy task.

The more important projects required explicit exploitation and re-modelling. That task came to us, and is ongoing as of Q1 2016.

Clearly security is very important with all the data in one place - so there is a lot of Kerberos, encryption and custom code in the BA server implementation here.

Once all the data is loaded the analysis possibilities become limitless. All of a sudden 360 degree customer view becomes easy as well as much more accurate reporting of business efficiency.

• Client: Major government organisation
• Date: Second half 2015
• Service: Full business analytics stack from ETL to analytics utilising Hadoop

Close

Community

---

The tech community in London is redhot right now with meetups for many different types of technology. The Pentaho community is no different and has been having a huge European wide meetup since the creation of the product. 2015 saw this event hosted in London, with Code BI, Ubiquis and Bissol Consulting the co-organisers. You can read about the event here.

Code BI also organises and frequently sponsors the Pentaho London Usergroup (PLUG). This is a quarterly event typically held at Skills Matter. You can keep track of upcoming events, and see the content from previous ones here. Additionally if you join the skillsmatter group you can see previous skillscasts too.

Finally Code BI has found that there is enormous value in donating time to help charities - and Bloodwise in particular. The story can be found here. What is particularly interesting about the setup at Bloodwise is that they are very similar to a lot of SMEs - using open source technology to keep costs down - and this is very effective. The charity oriented system they use is also open source, and also has an active community.

Close

Data 50 Awards

---

2015 saw our founder Dan Keeley listed in the top 50 data influencers in the UK. You can read the article here

• Date: 2015
• Service: Charity

Close