Credit Suisse saves £500k a month using Splunk, with Oracle database set to be decommissioned

Splunk has allowed firm to simplify its environment and better use its resources, claims head of grid operations and support

Financial services firm Credit Suisse is reaping the benefits of implementing operational intelligence platform Splunk, with savings amounting to about £500,000 a month.

That is according to Finnbar Cunningham, head of grid operations and support at Credit Suisse.

Cunningham told Computing that the company had initially built its own statistics platform, which it integrated into its in-house grid product to ensure it could monitor some of its operational jobs and see if it could better utilise its resources. The platform sent stats summary data back at the end of every job.

Users could tell the platform what analytics library to load and the necessary inputs and it would give them back the data they required.

"We had a component that picked up messages, wrote them into SQL and then we had a web front end that ran procedures, then used other components to display graphs and tables - essentially we created our own Splunk but it was not as flexible [as Splunk] for our users," he explains.

Credit Suisse eventually decided it wanted to extend the scope of its computing platform, and had to decide whether to develop the in-house solution, start from scratch in-house or opt for a vendor's product.

Cunningham had looked at Splunk as an option back in 2009, and says he was lucky that another team within Credit Suisse undertook all of the hard work of getting internal approval for the product.

However, he still had to run the proof of concept (PoC) for Splunk to justify its use and review it against other products.

Of the various other solutions that were looked at, Cunningham says the vendor that came closest to being chosen was Liquid Labs. But Splunk stood out, Cunningham says, because of the "quality of its product".

"Everything was easy, the search commands were really intuitive, the syntax was really clear and within hours I was doing stuff with data that we couldn't do with SQL - I was getting fresh insights. Previously I had to run a SQL query and if I was compiling a graph I would chuck it into Excel. With Splunk I could straight away get the graphs I needed," he says.

The PoC took place over a two-day period in December.

"A couple of us installed it, threw in some data out of SQL, set up a scripted input and we just grabbed data out of the SQL database every five minutes - and we immediately had a useful system. We actually used our PoC for a couple of months while I was reviewing competitors, and even after we set up with all of the competitors, everyone ended up using Splunk - so the PoC really confirmed it for us," Cunningham says.

Useful data

Cunningham says Splunk helps Credit Suisse to use its computing resources as efficiently as possible.

"We have data inputs which are grid statistics, so every time a job is completed, the grid produces some summary stats - the product has some of its own stats and analysis tools and uses Oracle on the backend, but I just grab the data before it gets into the Oracle database, it gets written to a file, I index that file and Splunk fills out all of the fields," he explains.

From a developer's perspective, this means that they can look at the performance or time it took to run a particular job pre and post changes.

"That is immediately giving us value. It means if they are doing something silly and it takes a lot longer, that would be wasted compute resources, which costs the bank money," Cunningham says.

The bank can also pull in process-level utilisation metrics such as CPU and memory usage on compute hosts, allowing it to look at utilisation from an operating system perspective for a particular grid application.

"That's when it gets really useful for us. In the old days we didn't have that view... What's great with Splunk is that we enrich our data," says Cunningham.

The company is also making savings in other ways.

"Because we've given our users direct access to that data, they are making decisions based on that data, making best use of the resources and they are saving money," he says.

Cunningham adds that according to his team's calculations, the changes are saving Credit Suisse about £500,000 a month in total, and it expects to save even more as it decommissions its databases.

"We had scripted input which was running a command line SQL query and pulling in data every five minutes, so that was an input from SQL. The grid scheduler uses Oracle. When I first set it up I was pulling [data] from Oracle but I have been working to completely eliminate databases," Cunningham says, adding that in some regions Credit Suisse has switched the Oracle database off.

But he downplays the amount of savings the firm could make by switching off its databases - stating that this is "miniscule" compared to the other savings.

"I think the best thing for us is simplifying our environment. Where we had to work with both SQL database administrators and Oracle database administrators, now we can just use Splunk. I've got a small team so anywhere I can save their time it is huge for us - it means we can do better things, spot inefficiencies and fix things," he says.