Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[multilingual-dh] open global multilingual television and online news annotation datasets

kalev leetaru kalev.leetaru5 at gmail.com
Wed Sep 28 05:53:39 PDT 2022


I thought many of you might find these global open data multilingual news
annotation datasets of interest.

In collaboration with the Internet Archive's Television News Archive, we
have made selections available of their entire global holdings, spanning 98
channels over 50 countries and territories in 35 languages and dialects
over 20 years:

https://blog.gdeltproject.org/tv-news-visual-explorer-announcing-98-channels-in-50-countries-in-35-languages-over-20-years/

https://blog.gdeltproject.org/tv-news-visual-explorer-announcing-the-emea-television-news-archive/

https://blog.archive.org/2022/06/02/a-new-approach-to-understanding-war-through-television-news-introducing-the-tv-news-visual-explorer-amp-the-belarusian-russian-amp-ukrainian-tv-news-archive/

Each news broadcast is sampled into a grid of images, one every 4 seconds.
The sampled frames are downloadable as a ZIP file to enable at-scale
non-consumptive computational research, enabling things like scanning for
how media from one country is repurposed by the media in other countries:

https://blog.gdeltproject.org/experiments-in-scanning-russian-television-news-for-fox-news-clips/

And 300+ language video OCR, for those interested in the use of onscreen
text across the world:

https://blog.gdeltproject.org/using-visual-explorer-googles-cloud-vision-api-to-non-consumptively-annotate-a-russian-television-news-broadcast/

We also have posted experiments on how state of the art commercial ASR
transcription performs on television news from around the world, including
unsolved challenges like code switching:

https://blog.gdeltproject.org/how-googles-speech-to-text-api-hears-television-news-from-around-the-world/

https://blog.gdeltproject.org/ai-challenges-speech-recognition-in-multilingual-societies-from-multiple-languages-to-code-switching/

For those interested in still imagery, we've analyzed around half a billion
global news images from around the world, including 300+ language OCR and
EXIF metadata extraction:

https://blog.gdeltproject.org/vgkg-passes-half-billion-images-top-labels-of-last-12-months/

And for those interested in textual multilingualism, we also have a
150-language realtime ngram dataset:

https://blog.gdeltproject.org/web-ngrams-3-0-tutorials-roundup/

For those interested in low-resource languages and language detection, we
also are rolling out across our datasets soon our first 400-language
detector:

https://blog.gdeltproject.org/translingual-2-0-a-new-era-in-globalized-language-detection-gdelts-first-inhouse-language-detector/

I thought a lot of these resources might be of interest! Email me if you
have any questions!

Kalev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/multilingual-dh/attachments/20220928/63a23dbe/attachment.html>


More information about the multilingual-dh mailing list