Amy Collins on Data: Introducing Four New AsiaPac Data Feeds Product

Product By Amy Collins on March 2nd 2017

In a recent blog post, I shared our plan to make Brandwatch the number one destination for social data covering the Asia Pacific region. In today’s blog, I’m going to update you on a data partnership deal that we recently signed and that our engineering team are in the process of integrating.

Most vendors of social data analytics solutions buy their data from a data aggregator who do their crawling for them. We are almost unique in the industry in that we do our own crawling. This is a strategic choice in that it gives us the ability to rapidly add new sites when asked and ensures that when we do it is available exclusively to our customers.

We have just signed a partnership with the main data crawling service in the industry. The theory is that by adding the data used by all of our competitors to our own crawled data, we will have by far and away the best coverage for the Asia Pacific region. Here are the projected data volumes that we’re seeing:

AsiaPac Data

1. News articles

The team has completed integrating the first of the four feeds that we’ve procured for you: News.

We’re already seeing the data in the app in the testing environment and you should start seeing it in the live version of Brandwatch Analytics in the next release (Thursday 8th March). The following is a really powerful illustration of the new AsiaPac news content from that test environment. It’s exciting to see the project results shown in such a visual way. We’re currently seeing volumes of between 5,000 and 7,000 news articles per hour.

AsiaPac Data

 

2. Blog posts

The second feed that we’re working on is blogs.

The team are making good progress and I will update you in a future blog when this is due to come online. Because it can be hard to identify where blog articles originate, we bought a global feed. In terms of volumes we’re seeing around 120,000 articles per hour.

3. Forums and message boards

Forums and message boards are high volume content: we’re seeing 600,000 per hour across the Asia Pac region. This will be the third stream that will come online in the coming weeks. Again, I promise to keep you updated.

AsiaPac Data

4. The full Tencent ‘Discuz!’ firehose

If you’re a westerner, you may not know of “Discuz!” or you may have assumed that I misspelled Disqus, the commenting engine (which we also provide in our platform). Discuz is a part of the Tencent ecosystem of social networks. If you don’t know Tencent, they’re a Chinese social empire to rival Facebook. Discuz is essentially message boards or forums that power vast numbers of Chinese communities. We’re seeing between 5,000 and 10,000 posts per hour coming in. We hope to move on to integrating this in the early weeks of Q2 2017.

The future of Brandwatch’s AsiaPac data

I hope to have some news in the next month or so about another significant Chinese data partnership – one of the large social networks.

Watch for a future blog from me.


Brandwatch Analytics

Open up a world of social data with Brandwatch Analytics

Find out more

Amy Collins

@amy_co106

Amy is VP of Product for Data at Brandwatch. Her responsibilities include developing the data strategy, data partnerships and acquisition and for owning the roadmap of new data products. She's a keen sailor and is mum to two little girls.

  • Giles Palmer

    nice job Amy

  • Anna Timofeeva

    This is definitely good news! Can you please share why News were put as priority for social media listening? And any plans to start tracking Weibo, Tencent QQ, Youku, Wechat?

  • Amy Collins

    Hello Anna,

    Was your question “Why was news the first one we integrated in the above list?” or “Why have we prioritized Asia Pac coverage over some other kind of news coverage?”.

    And re the big networks question… Like I said in my blog – I really hope to have something to announce on that front in the future as as soon as I am able to tell you – rest assured this will be the place I announce it.

  • Amy Collins

    Thank you!

  • Amy Collins

    Thank you Chief

  • Amy Collins

    Hello Anna,

    Was your question “Why was news the first one we integrated in the above list?” or “Why have we prioritized Asia Pac coverage over some other kind of news coverage?”.

    And re the big networks question… Like I said in my blog – I really hope to have something to announce on that front in the future as as soon as I am able to tell you – rest assured this will be the place I announce it.

  • Anna Timofeeva

    Hi Amy, I was interested why the news were prioritized over blogs and forums. Thanks!

  • Amy Collins

    Oh – good question. For a number of reasons:
    1. It was very slightly technically simpler than the other sources
    2. Because the volumes were lower, it meant that the first deliverable would be less likely to caught up in scalability complications during testing. If we started with the big one and we had challenges with volumes, then we’d have to work on that, delaying customer value getting out there. This way we start with one nice and small and even if there were scaling challenges with the higher volumes, you would get something to enjoy in the interim. For the record, it looks like there will be no issues with the scale, but I was just managing the risk
    3. Relative values depend very much on use case – there is no clean cut answer.
    4. It was a relatively trivial decision as the project is moving crazy quick, so if a is done before b, you’re only waiting a few weeks for b and another few for c. If it was months between, I _might_ have prioritised differently.

    Really appreciate your question – the job of a Product person is challenging – there are often no clearly right answers to any prioritization call. But I hope the transparency is useful.

    Have a lovely weekend
    Amy

  • i will copy this article on my blogs
    i think my webs can be popuar bookmarking site as above
    https://marrakechquadbiking.com/
    https://marrakechbuggybiking.com/
    marrakechcamelriding.com
    https://marrakesh-desert-tours.co.uk/

  • Adam Springer

    Thats a lot of data! :) Great article, very interesting numbers.