How to foster data driven culture

Last week I was talking to a founder who recently launched a product. He expressed frustration on how he’s the only one on the team who cares about metrics. His team values data in general, but didn’t end up tracking any metrics for their recently launched product.

I have been in this situation before and was able to guess several things that they as a team might be doing wrong. Over the years, working on several web and mobile products, I’ve learned so many things that I have been doing wrong to try to get the team on board. In hindsight, some of these things seem glaringly obvious but can easily be overlooked. Here are a few things I’ve learnt that can help small teams care about metrics.

Don’t Change Workflow

When working on product or marketing metrics is your focus, you’ll most likely have analytics tabs open at all times. Constantly checking numbers, looking at various reports, etc. Initially, to build a data driven culture, I passed on our Google Analytics credentials to everyone on the team expecting that they’ll login to check the numbers everyday. It was soon evident to me that that’s not the best way to get everyone on board with a metrics driven culture. There’s huge friction for other members of the team to login to Google Analytics (which was another Google Account), then click on the profile, click on the right dashboard and then see a bunch of numbers which they may or may not have any context of.

Two ways I found that worked best with bringing everyone on the same page about metrics are:

Email — It’s safe to assume that everyone on the team checks their email (or at least used to, Pre-Slack). We decided to send an email with our most important metrics everyday at the same time (preferably at the start of the work day) with a few key metrics in the subject of the email. Adding key metrics to the subject helped in other ways. More on it later in the post.

Slack —Several years later when I started using HipChat (and later Slack) we translated the success with email to Slack. Due to the (bite-sized) nature of chat/messages, Slack opened up more doors to embed lightweight analytics into the existing workflow. I didn’t want to send too many emails so we’d only send one email per day, but with Slack we could control the frequency and timing of messages as well as which channels they go to. We have a dedicated #reporting channel where a lot of metrics get pushed into. It’s more like firehose for metrics and reports. We avoided building several dashboards because we were able to push those metrics into the #reportingchannel. The #general channel only gets the most important KPIs.

Lesson Learned: In order to get your team to adopt something, treat them like your internal customers and come up with solutions just like how you would for the users. Try to incorporate as much as you can into existing workflows. Remove as much friction as you can.

Bite Sized Data

Being a product or marketing person, you look at several metrics from MAUs to engagement metrics. Although, these are great metrics to track, not everyone on the team needs to know about all those metrics. Once I learned that team members are usually not motivated enough to login to the analytics dashboard, I started taking screenshots from Google Analytics and sharing those with the team. Everyone looked at the numbers, but I could tell they were overwhelmed by all those different metrics.

When we started sending a daily email report, the biggest thing that I learnt wasn’t the email itself, but the Subject of the email. Here’s an example: “DAUs: xx,xxx MAUs: x,xxx,xxx”. The email body had a lot more details. I noticed that most of the time people would simply glance over the subject and deleted the email. Since these were daily emails, everyone had a ballpark idea of what normal metrics numbers were. The days when the numbers had a major dip or rise, everyone opened the emails to see what had changed.

Lesson Learned: It’s better to not dump all sorts of metrics on to the team and hope that they’ll weed out all the unnecessary info. It’s much better to selectively display 2–3 most relevant KPIs.

Real Meaningful Metrics

In the name of simplicity, it’s very easy to share feel-good metrics with the team but the main problem with that is that metrics like total registered users, total downloads or any other cumulative data numbers don’t really tell the whole picture. Leave those numbers for the press/marketing pages. Avoid using them as main KPIs internally. It’s good to share and celebrate cumulative numbers, but using daily/weekly/monthly numbers with delta change from the previous period can prove to be a lot more beneficial. One of the metrics we track and share on a daily basis is the percent growth or decrease in Weekly Active Users (WAU).

Monitor and Update KPIs

This is probably the simplest of them all, yet from my experience, ignored the most. Usually once the KPIs are added, they are never taken off the list. Gradually new KPIs keep getting displayed, but the old ones that are not relevant anymore are not taken off the list. It’s a good practice to clean up the list once a quarter. Get rid of things that aren’t relevant anymore.

Apart from a few metrics becoming less relevant, some KPI metrics don’t change too much. Instead of pushing the metrics every hour, push them only if there is a significant change. For instance, rather than blindly pushing hourly updates for average response time for your API, only push if there is a +/- 10% change. Showing same metrics often can make the team desensitized to the metrics (see banner blindness).

Best part of Slack? Custom emoji avatars and names for the bots!

Got tips or tactics on how to spread a data driven culture? Would love to hear them. Comment below or tweet at me.

About FourSquare

This post was initially posted on Medium as a response to Dennis Crowley’s interview with Steven Levy.

I think it’s a great interview. After reading this, I have been most bullish on FourSquare in years!

I was an avid FourSquare user early on, when it was launched. My Twitter and Facebook feeds were filled with my FourSquare check-ins from unique restaurants to next door Starbucks.

My usage started dropping even before FourSquare launched it’s auto check-ins, which were launched prior to the Swarm app. The biggest reason for my decline in usage was lack of product innovation for a long period of time. I’m not sure if the team was busy scaling or they were trying different things. I felt that the product was not evolving and hitting the check-in button over and over again got boring.

Auto-checkins came too late. Even when they were released, the product was all about what I input and didn’t give out much value to me as output. Check-ins and leaderboards gave me some entertainment value, but that’s pretty much it. At that time, continued check-ins felt like a hamster aimlessly running on a wheel.

A hamster running on a wheel.

I really liked this part of the interview:

I have to say that this wasn’t all that clear to me. I had to talk to you to figure this out.

I know, I know. We’ve never done a good job telling the story. We’ve never really embraced any forms of marketing and I think that’s a weird thing for a company that’s six years old. We just hired a VP of Marketing, and she’s had us run some outdoor ads, we are doing some digital ads, we’re sending emails now, we have much more targeted campaigns, we have a real strategy for how we’re gonna tell our story.

Knowing about FourSquare’s future vision could have got me to keep running on the check-ins wheel a bit longer. Being a founder, this is what I think a lot – are we doing a good job of telling our story and what the future looks like? Usually the answer is no, we can do a lot better.

The more I think about it, more I believe that even though marketing, press, outward communications help with telling and spreading the story, the biggest story teller for the existing users is the product evolution. Once the existing users buy in and trust the story, they will make it their life’s mission to spread that story. Apple & Android fans are a great example of users believing in and spreading the story. I wish, I could see the story early on with FourSquare’s product evolution.

Right after reading the interview, I checked my phone to see if I still have FourSquare app installed. Luckily, I was able to find it using the spotlight search and opened the app after several months, if not over a year. The app has changed since I last used it, which is a good thing that it is evolving.

Unfortunately, the story falls short here.

On opening the app, I am suggested these options.

Restaurant recommendations from FourSquare

I am not sure where these restaurants are. On opening the details I find out first one is in Oakland and second is in Palo Alto. Both these locations are 15–20 miles away from my current location.

Local Standouts on FourSquare for me.

All these places are about 15–25 miles away from my current location as well.

One thing I learnt about personalization over the years is that perception of personalization for users is more important than the personalization itself. This is why Netflix shows “Avengers recommended to you because you watched Captain America,” and Facebook shows “XYZ suggested as a friend because you have 23 mutual friends.” Just by knowing the reason for the recommendation, puts people to ease than not knowing.

I know FourSquare has great location data and potentially a kick-ass recommendation algorithm under the hood. However, when I see these results, I don’t know why are any of these places recommended to me. It is possible that I have lot more check-ins in Mountain View/Palo Alto area than where I am and this is why I am shown places in the Peninsula. It is possible that when I setup FourSquare I told them I like Mexican food and desserts. Being a user, I don’t remember any of these things and would love to understand why am I recommended these places.

I think Yelp needs a good competitor. FourSquare has the potential to be that. I hope FourSquare gives me more reasons to use their app again.

Why Slack?

4 Replies

I have been running remote teams for almost a decade now. Tools available for remote teams have evolved a lot during this time.

In the early days tools like Skype made it easier to communicate with remote teams via text, audio and video. After that came a wave of tools like Dropbox, Google Docs that helped manage files in the cloud for remote teams. However, these utilities were never able to create a real office like environment. A real office doesn’t feel like you are transacting with a co-worker, but has a character to it. You don’t talk with co-workers only when you need something. You overhear other people talking about the new episode of The Game of Thrones. You crack jokes, you huddle. Not all these things can be done by remote teams, but having a “virtual office” can help get the team connected better.

HipChat

At RightBuy, we started using HipChat, which is an excellent chat application for a small team. I really loved the simple design. We kept things simple by just setting up 1 room where our small team interacted. HipChat had opened up their API and enabled teams to integrate with external services like GitHub, Pingdom, CircleCI, etc. using Zapier. That was pretty cool. I have heard CampFire had some of these great features, but never used it.

Slack

I found about Slack through Jason Calacanis‘ interview with Stewart Butterfield. It was intriguing, but I dismissed it because it appeared to be exactly same as HipChat with IRC-esque UI. On asking other startups using Slack about differences between HipChat and Slack, all I got was – it has built-in search and it is pretty cool. I tried asking Slack themselves about why people are going crazy about Slack, they didn’t have anything concrete to offer either.

Slack Replies

Slack Replies

Search wasn’t a big deal for us so that was the end of Slack’s evaluation.

Slack Re-evaluated

About a month later, a friend from Seer was going gaga about Slack and showed how they were using Slack. That was the moment where I actually found out how Slack was a lot more than HipChat. The thing that sold me on Slack was how well Asana is integrated with Slack. It is a first-class integration that updates any comments or new tasks. As compared to this, HipChat had integration using Zapier and had limitations where we only could get only new tasks from a specific project. To keep things simple and extendable, HipChat didn’t provide custom experience for these integrations. I don’t mind using Zapier for integrations, but the UX on HipChat’s side was lacking. Here is how the integrations (at least for us) looked in both of these services.

HipChat Asana Integration

HipChat Asana Integration

Slack Asana Integration

Slack Asana Integration

Asana isn’t the only better integration. GitHub, Sentry and the webhook to push messages into Slack can be customized a lot more and provide a better experience. For instance, this is the same commit from Github to HipChat & Slack.

HipChat-Github Integration

HipChat-Github Integration

Slack-Github Integration

Slack-Github Integration

Notice Board

I found out from Bryan Helmig that they use something called async at Zapier, which is an internal WordPress blog for things that do not need immediate attention. It seemed like a great idea when I heard about it, something that I would like to implement for RightBuy as well. We have a lot of conversations among us that are way too long for ‘chat’, not actionable enough to be put in ‘Asana’ but at the same time needs to be captured in some shape or form.

Slack has a great feature for this use case. You don’t have to setup an internal WordPress blog, but just use their ‘Post’ feature. Anyone can write a post which can be shared with any channel or a person. It is great to capture things like your medium to long term goals, certain processes and in general company updates (especially if you are a big team.) The posts act as a ‘notice board’ (when was the last time you actually saw a notice board?) for the company.

Another great feature Slack has that I hadn’t even thought would be useful until we started using Slack – Favorite a Message. It is a like bookmarking a message for easy access for later on.

Not Everything Works

As I mentioned above, I really like that they are themselves building the integrations with other services. It provides a unique experience which can be customized based on each service they are integrating with rather than have a one-size-fits-all model where none of the integrations have an outstanding experience. That being said, not all of their integrations are the best. For instance, we found out that Slack’s default RSS integration is far inferior to Zapier’s Slack-RSS implementation. I haven’t tested all the integrations, but wouldn’t be surprised if there are more integrations that are being implemented better by third parties.

Another nitpick with Slack is that when writing a ‘post’, they use Markdown syntax. I would much rather use a WYSIWYG editor that can speed up formatting.

Slack Post Markdown

Slack Post Markdown

Email Killer & Future

I remember sending a lots of “chatty” emails to my co-workers a few years ago where everyone on dev mailing list had to bear with them. Combination of tools like Asana and Slack are effective on killing email use at least internally for small teams. This is a great start. Slack seems to be integrating with several services, but it has to be seen where does their future live. To expand, do they in future start competing with likes of Dropbox, Google Apps, Skype, or even Asana and provide a plug and play infrastructure layer for remote teams for a virtual office? It is yet to be seen, but so far they seem to be heading towards the right direction.

Did I miss something? Would like a follow up on how we use Slack? Reach out here.

How we reduced bandwidth by 4X

Leave a reply

One of the sites I run has been growing a lot lately. Late last year as the traffic grew, page load times started going up. We already had a lot of things optimized, it runs on a server a bit more powerful than my Macbook Air. I had tried Cloudflare in its early days and had complaints from several people that it was very aggressive with its security captchas. I gave Cloudflare another try (with low security settings) and the results were phenomenal. Not just our page speed increased, it just dropped our bandwidth to almost half. Since November we have increased our traffic 2.5X and as you can see bandwidth started increasing back again in February. This is when I setup minimum expire TTL (in Cloudflare settings) to 1 Year and that gave us another drop in the bandwidth usage. This helps with the returning users, not so much with the new users.

We have increased our traffic around 2.5X and reduce the bandwidth to half (perhaps more). If someone is not using Cloudflare’s free CDN and other features like auto minify JS, CSS and HTML they are missing out. No matter what the traffic is, your users and server will thank you.

We have also experimented with Varnish and a few other things, but haven’t been able to set it up with a very high cache rate yet. More on that later.

Entreporn – NSFW

2 Replies

A couple of weekends ago I was coding at TechCrunch Disrupt’s hackathon. This was my first full 24 hour coding hackathon and it was a lot of fun. While taking a break, I had pizza in my one hand and other scrolling down an article about how some guy shut his company down and lessons he learnt. Harish, who was sitting next to me pointed out – “we are reading the same thing!” I looked over his screen and indeed we were. He mentioned that he found the article on Hacker News and started talking about that particular article and in general content like that. We talked about how fluffy most of this content is and doesn’t add much value to the user, just like porn. He was quick enough to coin the ‘Entreporn’ term.

It is true.

Hacker News

Most of the articles – lessons learnt, “10 things you can do to get X”, etc. – just like porn are pieces of content that make one excited and give a feeling of achieving something by reading other’s stories. But the truth is that these articles don’t add much value to a startup founder’s life (this blog post included.) You don’t accomplish anything by reading about other’s experiences other than getting few ideas which you may or may not remember when you are actually in that situation.

Some of you may disagree. You might provide a few instances when you read something related to entrepreneurship and it changed your life. Actually, I myself have got a lot of value out of reading the JFDI blog post by Mark Suster. There was a time (back in 2007) when I was obsessed with reading TechCrunch. I made sure I don’t miss ANY story posted on the blog (they used to write 5-6 posts back then). More recently, I have also been checking my Twitter stream, Hacker News and other sources often a day and spent a few hours reading Entreporn because it gave me insights into things and supposedly made me a lot more prepared to work on my startup.

You bet I was wrong.

These feel-good articles do add value, but the value added by reading these articles is far less than by actually doing things and putting that time & energy into your own startup. Once I started ignoring most of this content, my productivity level increased a lot. By choosing not to read all the interesting articles, I don’t feel like I am missing out on some great advice or some snazzy marketing tactics. I only read articles/blog posts that I come across at least 2-3 different sources without making much effort. This helps me clear out a lot of noise/fluff and get to the important content only.

For all those people reading this, try cuting down on Entreporn (like this one) for a week and notice the change in your startup’s productivity.

Got any other productivity ideas? Would love to know about them in comments below or HN.

Kevin Rose's Foundation

Top 4 entrepreneurship podcasts for busy startup founders

1 Reply


One can read several blogs, scan people’s tweets, go to conferences, listen/watch tons of podcasts to learn insights into how other entrepreneurs operated. Who doesn’t like to get free advice from people who have been there done that, but the reality is as a startup founder you always have a shit load of things that you wanted to get done yesterday. Below is list of podcasts that I subscribe to (with my comments on them). Hope it helps you find good content and save time. Continue reading →

How to communicate with “Regulars” about your startup

Leave a reply

It is ironic that an introvert like me is writing about communication, but here I am.

While working on a startup you meet a lot of other people who are in the same boat or have passed through the phase you are in. At the same time you also meet a lot of people who haven’t heard of Eric Ries, Steve Blank, Convertible Debt, Pivot (maybe they did), MVP, AARRR, Lean Startup and the list goes on. I call these “regular” people or the sane ones. They could be your friends, neighbors, parents of your kids friends, mailman, teller at your bank and sometimes even your own family members.

When one of the “regulars” ask me about what I do, I start thinking about what version of my pitch do I tell them. I used to mention the current “solution” of the problem I was testing with my startup/product. Some times people questioned back about it and asked more details, but usually I got the “OK. Sounds good” response. I believe “OK. Sounds good” is the one of the worst answers to get from a person you are talking about your product. It is most likely think it is stupid or worse they didn’t get what you are doing.

One of the other things about telling “regulars” about the current “solution” you are experimenting is that it is an experiment and most likely will be replaced with another one in a few weeks, if not days. When you meet the same “regulars” again and talk about how you are doing this new experiment now, in their head they validate that you are a loser, you have no idea what you are doing and your product is going nowhere.

A lot of founders say that they don’t care about what people think or say about them or their product as long as they are on the right path and see progress. I disagree. Being a Founder/CEO your job is to make sure that from your grandma to your neighbor’s 5 year old boy should understand what you do to a basic level.

I started testing a few things with “regular” people about 3 weeks ago. I started talking to them about the problem (not the solution) I am trying to solve and in some cases I ask them even before telling what I do about how they find the products they want to buy. How did you know which camera to buy when you bought the last camera? Why did you buy your last camera? These questions are very relevant for RightBuy and help me understand the user’s thought process and current buying/researching habits. Talking about the problem helps me understand how many people have the problem vs. how many don’t even think it is an issue. Earlier, when I talked about what I am trying to solve the conversations went into silence because I talked to them about something they never saw, but something that I live 24/7.

Since then I have noticed a lot of people getting passionate about the problem and talking in detail how they wasted a lot of time and if they can use the application right now. There are also people I meet who haven’t experienced the problem or don’t consider it as a problem. Both kind of people help me understand who can be beta testers for the app and even evangelists.

I would love to hear other ways you talk and engage with consumers/customers who are not early adopters or tech savvy. What kind of questions do you ask them to get honest, constructive, actionable feedback?

Early thoughts on bo.lt

5 Replies

bo.lt is a service that let’s you copy, edit and share webpages. TechCrunch labels bo.lt as “bit.ly on steroids”.

The core copy and edit functionality looks really solid and works well. It finds all the images/css/js and other dependencies of the page and replaces them in the page and serves it up from their own S3 account. They have done a good job with the design as well.

As a user, I can think of times when I would want to use this service, but not as much as I use bit.ly. Most of the time when I share links, I don’t need to edit the pages.

As a content publisher, I am not too sure if I want users to use this. I understand that anyone can copy my content, edit it and host on their servers right now even without bo.lt. I believe bo.lt makes it easier for content publishers to lose control over the content. Any future changes to the original content will not be passed onto users with bo.lt.

According to TechCrunch: “…is content provider friendly in that Bo.lt still serves up a given page’s ads and analytics systems.” I haven’t tested the ads, but bo.lt does seem to append their own Google Analytics code to the existing GA code on the page. I am not sure how they work with other third-party or home-brewed analytics systems.

Another issue as a content provider is that bo.lt now competes with the original page for Search Engine rankings. For instance this page competes with  the original page without any reference to the original page. Currently, this whole model is breaking the web as we know it to multiple versions of almost same content with different URLs. Depending on how important getting indexed by Google is in their strategy, it can be easily fixed by either adding ‘noindex’ to bo.lt pages or even better adding a canonical tag to the original page. Perhaps, this is more of a reason for content publishers to have a rel canonical tag on their pages so that they get credited for their content.

It takes a lot of effort to change elements of a webpage on the fly and expect most of them to function as the original one. I think bo.lt is an impressive technology, but I am not too sure that they will be well received by publishers with their current model. I hope that they evolve and make it compelling for publishers to be comfortable with their product. They have raised $5 Million from Benchmark Capital.

Update 1: Brian Rutledge mentions that bo.lt seems to be moving rel=canonical tag into the body section, making it obsolete. However, that’s not what my experience was. Looks like a bug in bo.lt.

Update 2: I wanted to see if there is a way a bo.lt can be blocked to copy a website, same way crawlers can be excluded using robots.txt. I found that bo.lt does not look for robots.txt prior to grabbing a page. I looked through my apache logs and it never requested robots.txt today. Another problem is that they are faking the User Agent to be Firefox 3.6.4 (see image below). Whois on the IP address confirms it is owned by Boltnet, Inc. The only way to stop bo.lt from copying your pages is by banning IP address 199.204.84.2, until they change it.

Stop using “hits”

Leave a reply

Back in the days of early web (pre-bubble 1.0) “hits” was a popular metric used to measure popularity of a webpage/site. Technically a hit is “a request for a file from the web server.” Any file – jpg, png, css, js, pdf, html…you get the point.

I am amazed how many people still use “hits” when talking about their website and even tech bloggers use them in their articles. It is not that the hits metrics is wrong, it is useless. It does not tell you anything about the actual usage of a website. I can always increase the number of hits by 10X my blog gets by adding 10 1px by 1px images in the footer. The number will sound really larger, but the usage of the website will be same.

Now that we know that the 2,200 – no – 22,000 hits Zuckerberg got in the movie The Social Network isn’t that impressive, let’s see how should we refer our traffic as.

I believe when most of the people these days talk about hits, they actually mean pageviews. As a tech startup founder or a tech blogger you need to know what is the difference between a pageview and  hits.

I consider “pageviews” as the lowest granular metric that you can talk about a website. It is better than hits, but still doesn’t tell you much about the user retention or engagement.

Here are some of the metrics that do tell something about your website:

  • Visitors
  • Unique Visitors
  • Repeat Visitors
  • Pages Per Visit
  • Time Spent Per Visit
  • Monthly Active Users
  • Daily Active Users

All Tech bloggers and startup founders – Please stop using the word/metrics “hits“.