WebPerf How The Next Web Scaled to 7.2 Million Visitors a Month

MaxCDN is proud to present #MaxScale, our new series that takes an in-depth look at how growing tech companies handle scale at the highest levels of bandwidth, storage, analysis and more. In this installment we chat with Boris Veldhuijzen van Zanten, co-founder of The Next Web that uses MaxCDN as its content delivery network.

The Next Web was founded in 2008 and has become one of the world’s largest online publications that delivers the latest news about Internet technology, business and culture. The company, known by many as TNW, is a leader in the online publishing industry with 7.2 million monthly visitors and 9.5 million monthly page views.

TNW also hosts The Next Web Conference, an annual Internet conference where startups, speakers, and attendees come together from all over the world to talk about the future of the web. The conference started in 2006 with roughly 300 attendees and has grown to 3,500 expected attendees in 2015.

In this interview, Co-founder and CEO Boris Veldhuijzen van Zanten shares with us some of the things he’s learned from creating, growing, and scaling The Next Web.




Boris Veldhuijzen van Zanten, CEO of The Next Web and member of the Startup Europe Leaders Club, an organization devoted to creating a better startup climate in Europe.



Chris Hardee: When you first started the blog in 2008, you were up against some major competitors in the tech publishing space like TechCrunch, ReadWriteWeb, GigaOM and Engadget. At the time, were you planning on going head to head with sites like these, or anticipating the type of growth you’ve seen in the past seven years?

Boris Veldhuijzen van Zanten: No, we didn’t expect the growth to be as big as it is. When we started, we didn’t anticipate, plan, or strategize to compete head-on with these other blogs. We just felt that there was something we wanted to say. We wanted to say it in our own way, and we thought there was an audience. We didn’t really think about the size of the audience, but thought there was an audience interested in the same stuff we are.

It’s like when you throw a party, you’re not thinking about how many other parties there are; you just think: “I’ll throw a party and invite all my friends and it’ll be fun.”

CH: Can you walk me through the technology stack for your site, and how you chose each major piece?

BVZ: Yes, it’s a regular LAMP stack. We run WordPress and we spend a lot of time optimizing the servers and caching to make the site as fast as possible and always available for our readers. We’ve done a lot of optimization and customization on the WordPress install, and we’re very ambitious in going even further. One way we want to compete with our competitors is on innovation. I think this year you’ll see a lot of new developments on the site, and there’s some stuff you won’t see as a visitor. But there are a lot of steps we’re going to take to make publishing more dynamic and interesting for our readers.




The Next Web has helped lead the tech publishing industry since it was established in 2008.



To give you one example, it’s very interesting how publishing has changed over the past 100 years. Where 80 years ago if you wanted to become a publisher, you started by chopping down trees for paper to build a factory, or you installed expensive hardware that you couldn’t even lift. And if you wanted to distribute worldwide, you had to be a millionaire before you even got started.

Nowadays, there are still professional publishers. But for these professional publishers it’s more difficult to get their content seen by a lot of people than it is for amateurs. So if my mother sees a site that she likes, it’s a copy-paste to Facebook and she’ll press the return and she has published a review. And the whole process is automated. She doesn’t have to make a screen shot, she doesn’t have to write HTML, it’s just instantly there, including a preview where you can click it and a lot of people can comment.

With WordPress, there’s an admin dashboard. Facebook doesn’t even have an admin page or dashboard. The content creator and viewer are both seeing the same thing. That’s an interesting difference in how my writers must get involved with HTML, because they have to know how to handle images, align them, add tags, add categories, and a lot of other stuff before they can publish. So we’re trying to break through that and make it easier for editors to publish, but also add the kind of metadata that’s interesting for readers.

​

Usually, when an editor writes a story, his real value to the story is his analysis or his opinion or his verdict on something. But for most publishers, that’s like ten percent of the work. The other 90 percent you’re doing the introduction text and other stuff around the post. So this is something that we’re working on and you’ll see happening this year.




Collaborative workspace at The Next Web offices in Amsterdam.



CH: According to your media kit, you’re reaching 7.2 million unique visitors per month. At that level of traffic, what technological challenges exist that small publishers wouldn’t be expecting?

​

BVZ: Oh, many. It’s very interesting how you can start a blog and grow to a lot of visitors pretty quickly. But there comes a moment, and it’s probably between 500,000 and a million uniques a month, where you really need a dedicated developer to make sure everything runs smoothly. I think above one million, there’s no way you can just continue with off-the-shelf open source software. You have to go to customized stuff. You have to spend a lot of time on caching and optimizing the code, minimizing scripts, etc. That’s a constant struggle. It’s not something you can do once and then be done.

CH: What are three hard-to-spot pitfalls that are critical for growing publishers to avoid?

---

BVZ: 1. Be patient.

Number one is that you have to be very patient. It takes some time to become part of a reader’s routine. There are a lot of sites out there and a lot of people start writing. I meet hobbyists and professional publishers that are always negatively surprised by the amount of time it takes to become sort of a household name, before people will get used to your presence and start considering you as an integral part of the ecosystem. You have to be patient and realize that it takes longer.




The Next Web offices in Amsterdam, Netherlands.



2. Keep up with technology.

I think the second part is that the technology is an integral part of being an online publisher. Sometimes I talk to publishers, especially more old economy publishers, who see technology as a detail. In their minds, “it’s just got to work.” But the reality of being an online publisher is that the technology is changing so fast. By being an early adopter of technology, that’s how you stand out from your competitors. So if something new comes along, we as a tech blog generally know about it before it happens. The moment it becomes available, we’ll have it implemented. We’re already thinking about how we could use the Apple Watch in our publishing routine.

​

We’re interested in that because we have a passion and interest for these things. Nobody’s telling us that we should look at it; it comes naturally. Then I speak to our competitors, who say: “We’ll see what happens there, but let’s give it a year because we’re not sure if it’s going to be anything, plus we don’t want to invest unless it becomes important.” The downside of having this approach is that you’re always late to the party.

3. Focus on great people.

I think the third one is that publishing is a people’s business. It’s fairly important that you have great writers with insight and personality, where in the past, publishing was more fact-based. Something happened, you reported that it happened, and that was enough for the readers. I think nowadays there’s too much competition. For example, when the plane in Taiwan went down, I saw it on Facebook first then checked my local newspaper. And it took like two hours before they had something up about it.

Just reporting the facts is not enough anymore. The background of the story (the what’s in it for me), the analysis, and the opinion are much more important. That’s the added value of publishing now. That means you have to find people who are smart, opinionated, and able to explain to a reader what the facts actually mean. So focus on great people.

CH: Looking three to five years into the future, what do you think will be the next big change in tech publishing?

BVZ: If you look at the added value you bring as a publisher, it’s not just the facts. I think once you realize that, it makes sense to have more room for the extra value, the metadata that writers add to a story. I think that will influence publishing greatly. That’s more content than technology.

If you look at technology, one of our visions or our strategies is that we always anticipate more technological change in the future than we’ve seen in the past. We think that technology has a tendency to speed up. As more new technologies are invented, more unexpected combinations of different technologies become possible. That’s very inspiring. It can also be frightening. But as soon as you start anticipating that, it becomes the status quo.

​

There will always be change and new opportunities. You’ve got to be ready for that. What you’ve got to do is make changes every day. Optimize for new things, always improve as the world is changing and new audiences start looking for your traffic. That’s how we look at things. We keep a close track of our statistics, and if we see growth somewhere and we see people using our content in different ways, then we start optimizing for it.

The other way is we find new technologies and actively try to see how we can take advantage of them. The Apple Watch is a great example. If there’s any way we can give people our content on an Apple Watch, we’ll certainly try to do that. Not because we expect it’s going to change everything, but it’s just another way that we want to reach our readers.

CH: What is the most complicated technological problem The Next Web solved this past year?

BVZ: I think that’s a really good question, and there are two answers. Caching pages, that’s the first thing that pops up. I think for most companies the modus operandi is to see technology as being stable, then solve problems as they arise. For us, it’s more like we see our technological infrastructure in a permanent state of panic, where we are constantly looking for the next problem that we’re going to get, and how we can prevent it.

Other things to consider are:

• What part of the process is at the top of our list for taking the most time, and how can we optimize it?
• What is the page fill time and what is the slowest script, and how can we optimize that?

Our day to day operations are more filled with problems or challenges and improvements than all green lights.




Amsterdam workers enjoy a comfortable setting while collaborating with the many remote workers associated with The Next Web, including staff located in Thailand, Nashville and Dubai.



CH: There are quite a few tech companies and publishers who are getting into the event and conferences market. The Next Web went the reverse route and began publishing after starting the events division of The Next Web. What advice would you give to those companies just getting into the event production side of the business?

​

BVZ: You organize the best event you can organize, and that seems logical. But I meet publishers who just ask how do you organize an event? Or do you go to another event and try to copy it? Or they’ll try to find a template for organizing great events.

I think it makes more sense, and it has always been our strategy to think: “What would I like in an event? What would be the event that I would like to bring the public to?” Instead of looking at the bottom line first, I think this is a better long term formula for success than just looking at it as a revenue model where you try to squeeze out as much money as you can. If you have long-term vision, it makes sense to make an event that doesn’t look like one everybody else is doing. Not like the event that your competitors are organizing, but something that really fits with your audience. They will reward you for it in the long term.




Co-founder and CEO Boris Veldhuijzen van Zanten at the office.



CH: If you had a list of best-kept secrets about scaling and growing an online publication, which ones would you include and why?

BVZ: I think a well-kept secret is how much incredibly hard work it is to be an online publisher. I think some people think: “Oh, that’s great, I can just sit on my couch and write some stories.” But we’ve had writers in the past who started and after a month said: “I’m sorry, this is just not the life for me; it’s too much work.”

​

I think another well-kept secret is: Nobody knows what they’re doing. The interesting thing is when you start, you think: “What am I missing?” What does everybody know that I don’t know? And then it takes a few years to realize that for everybody, this is a new world. The rules are new, the audience is basically changing and there’s no fixed set of rules. There’s no scenario for success. You kind of find out on your own and have to reinvent the wheel every day. The interesting thing is that we don’t come from publishing. We’re geeks and nerds first. We saw an opportunity in publishing so that’s how we got started. And so for us, this constant change is more exciting.

When I speak to people who come from publishing, they seem to see the technology more as a threat, as a disrupter, as something that’s giving them a headache. And that’s a huge difference that you can take advantage of if you understand that your competitive edge can be technology. If you see it as a positive part of your work, then the sky’s the limit.



Photography by Stephanie Boonstra. All images released under Creative Commons license. Please attribute photos to “Photographer: Stephanie Boonstra / Source: MaxCDN” and link to this post.

The post How The Next Web Scaled to 7.2 Million Visitors a Month appeared first on MaxCDN Blog.

Continue reading...

 

Well yes properly configured Nginx server can easily replace 2-4x Apache servers so can scale much better with Nginx. Centmin Mod Nginx is closer to that 'properly configured' end

 

Was thinking about @deltahf's question a bit more and there's other factors too

1. Server hardware has gotten more powerful and scales better clock for clock over the last few years. You can handle alot more one current Intel Xeon generation of single and multiple processor supported platforms. For instance, Intel Xeon E3 v5 Skylake is easily 30-85% faster than Intel Xeon E3 v3 Haswell for encryption/decryption. More cpu cores, larger and faster processor caches and faster DDR3/DDR4 memory, SSD and network capacity. You can get up to 56Gbs network connectivity more easily these days and sooner it will be more relatively affordable with 25Gbps and 50Gbps networks too = We'll have 25Gbit & 50Gbit ethernet network connectivity options soon !
2. Software stack has improved tremendously for PHP 7 being 2x times faster than PHP 5.6, MariaDB 10.1/Percona 5.6/5.7 and Oracle MySQL 5.6/5.7 leading the way. Nginx getting better and better with support for reuseport, HTTP/2 and soon ngx_brotli compression
3. Intel/Micron 3D Xpoint Memory will be an interesting technology too Intel/Micron 3D Xpoint Memory Storage !