Artwork by Khyati Trehan

How Wealthsimple Trade Didn’t Break During GameStop

Wealthsimple makes powerful financial tools to help you grow and manage your money. Learn more

Yes, there was baking and Netflix. But do you know what else has gone insane in the past year while everyone’s been trapped at home? Trading stocks. So many people, so many stocks, so much exponential growth.

It was a seismic change, but it was focused on one area: this thing called retail investing. Meaning normal people who don’t work in the financial industry and aren’t trading on behalf of hedge funds or mutual funds or ETFs — just regular investors, sitting at home, buying and selling shares of companies they like.

How do we know retail blew up so much? Because we run Canada’s fastest-growing retail investing platform. Wealthsimple Trade, our commission-free stock-trading app, grew by about 1,000% in the past year. And then in just the first six weeks of the year, Trade grew by another 400%. But unlike many platforms, Trade didn’t experience any outages in trading during that time.

As people who are interested in the intersection of technology and money, we got excited to dig into exactly why that was and how you re-engineer a company in the midst of a tsunami of new users. So we called up Karney Li, Wealthsimple’s Chief Technology Officer, and Dominique Simoneau-Ritchie, VP of Engineering, and asked them how it all worked.

We knew they were coming. Sort of. As of January, the engineering team at Wealthsimple was expecting growth — because we were already growing. We had a big spring last year, grew more during the early days of lockdown, and the number of Trade users spiked even more dramatically in November and December.

Meanwhile, the engineering team was also looking ahead to February 7, when an ad for Wealthsimple Tax would run during the Super Bowl. They knew from experience, there would be a surge in demand for our services across the board. “We started planning for 10X growth,” explains Li, adding, “Because when you increase by an order of magnitude, that’s when things really start to break.”

In engineering terms, that meant accelerating a transition to “real-time event-driven architecture,” says Simoneau-Ritchie. “This means our systems are able to complete many client requests at the same time, rather than in sequence. And that, in turn, allows us to respond to our customers much more quickly — usually instantly.” In November, for example, the team made a change (for actual engineers out there: they moved a key microservice to an event stream instead of polling), and that led to an 87% increase in how fast Wealthsimple can execute trades when the market opens.

Engineers from across our teams also worked together to brainstorm creative ways to reduce the number of requests across our microservices. The engineers noticed something interesting: many people tend to hold, and watch, many of the same securities. Noticing this pattern, we installed something called Varnish, which is an HTTP reverse-proxy, a web-accelerator (it’s ok if you don’t know what those are, we’ll explain). In engineering terms, Varnish intercepts requests — and, importantly, it can coalesce similar requests, which saves a trip to the server. That strategy allowed the team to reduce the number of actual requests to the server by over 99%. Varnish is also really, really fast. Say a lot of clients are requesting quotes for Air Canada, all at the same time — now the system can fetch that data once and service all the requests with much less delay.

Then the meme-stock phenomenon happened. In mid-January, the GameStop phenomenon — aka the meme-stock bubble — began to sweep the markets. Overnight, Trade was experiencing growth that made 2020 look slow by comparison. In the weeks after New Year’s Day, the app’s user base quadrupled, and we faced a new problem: how do we handle all these new customers?

“In the first week of January, we knew we could handle 9,000 clients concurrently,” Li says. That means our technology was stress-tested to handle, on average, about 9,000 users making trades at the same time. “But now all of a sudden we had to sustain 25,000 clients concurrently.” Engineers generally don’t feel comfortable unless their systems can seamlessly handle twice their normal “spike” capacity. But Wealthsimple’s systems were now facing three and four times that much activity on a regular basis.

So how exactly do you scale a tech product that’s blowing up (in the best possible sense)? According to Li, there are a couple of ways. The first is vertical scaling: that just means going bigger. You increase the size of a server by getting a bigger machine, bigger database, faster processor, what have you. “It’s like a bus,” he says. “You go from a single-decker bus to a double-decker bus to maybe a triple-decker bus — it’s still taking up the same amount of physical space but you can move more people.” Throughout our run of growth in 2020, we did plenty of vertical scaling.

Sooner or later, though, you can’t make the bus any bigger. You need more buses. More hardware. This is horizontal scaling, and it’s what allowed us to keep pace when everything went bonkers during GameStop. “In a couple of days, we doubled the number of containers we were running behind the scenes, from approximately 1,000 to about 2,000,” says Li. “And when you do that, your bottlenecks become very apparent. We were able to see where things were getting held up, processes that were really slow. This allowed us to keep up with the ever-increasing number of concurrent users.”

“We were all-hands-on-deck in a virtual Zoom any engineer could join to watch and learn,” says Simoneau-Ritchie. “And when we were increasing capacity, a handful of our systems might need to go into maintenance mode for a few minutes so we'd plan to do that at night to minimize disruptions to clients.”

“It was very stressful,” Li says. “But also pretty exciting.”

And in the process of solving these problems, we solved other problems we didn’t even know we had. Prior to GameStop, our team mostly focused on ensuring quality performance for people simultaneously using the app. Now, for the first time, we started to see signs that people might be experiencing slowness when they were simply trying to sign up. When our engineers looked into it, they isolated a pain point we hadn’t noticed before: we were generating PDFs during the user sign-up process that could simply be generated later. And when we fixed that, we saw a significant uptick in performance. “Those types of optimizations have huge impacts on our ability to onboard new clients,” Li says. “As a result, our trading actually got faster during the GameStop period. Though, of course, there are still moments of slowness. We’re working on those, too.”

The whole thing reminded Li of a favourite aphorism of his: never waste a good crisis. “Some of our issues were long-standing problems that probably never would have gotten fixed if we did not have this crisis,” he says. “And when you go through stressful periods like that, the company and the engineering team learn a lot.”

Wealthsimple uses technology and smart, friendly humans to help you grow and manage your money. Invest, save, trade, and even do your taxes in a better, simpler way.

Money Diaries

"I DON’T KNEEL DOWN AT THE ALTAR OF THE ALMIGHTY DOLLAR. THAT’S NOT FOR ME."

Spike Lee Tells Us Why He Never Feels Bad Asking for Money

subscribe

Get the best stories from our magazine every month

Sign up for our email newsletter

  • News

    Building a Better Wealthsimple: A Diversity & Inclusion Update

    It’s the start of a new year, and it’s a good time to check on what we’re doing to make good on the promises we made last year to be a more diverse, inclusive company. Here’s where we are.

  • News

    The Free Government Money Isn’t the Only Reason To Open an RESP

    All hail the Wealthsimple RESP. Now that we’ve eliminated the confusing paperwork and high fees, there's no good reason not to start planning for your kids' future.

  • WEALTHSIMPLE TRADE

    Commission-free stock trading

    Wealthsimple Trade makes it easy to buy and sell thousands of stocks and exchange-traded funds.

    see-more cta
  • News

    Wealthsimple Has the First Regulated Crypto Trading Platform in Canada. But What Does That Even Mean?

    Wealthsimple Has the First Regulated Crypto Trading Platform in Canada.

  • News

    We Were Not Living Up To Our Principles. Here's What We Are Doing About It

    We believe Black lives matter. We are committed to building a more diverse team, and to supporting our Black communities.

WEALTHSIMPLE TRADE

Commission-free stock trading

Wealthsimple Trade makes it easy to buy and sell thousands of stocks and exchange-traded funds.

full cta

Our best stories, once a month.

Sign up for our newsletter

The content on this site is produced by Wealthsimple Technologies Inc. and is for informational purposes only. The content is not intended to be investment advice or any other kind of professional advice. Before taking any action based on this content you should consult a professional. We do not endorse any third parties referenced on this site. When you invest, your money is at risk and it is possible that you may lose some or all of your investment. Past performance is not a guarantee of future results. Historical returns, hypothetical returns, expected returns and images included in this content are for illustrative purposes only. By using this website, you accept our (Terms of Use) and (Privacy Policy). Copyright 2021 Wealthsimple Technologies Inc.

;