Basic - OSINT - M04 - Gathering data from Facebook

This post is a part of the OSINT series.

1. Why would you want to use Facebook in OSINT?

I think that Facebook does not need any introduction. In simplest words, Facebook is the most popular social network / platform on the world at this moment. You can probobly see Facebook logo everywhere, on every website, on most products etc. This portal is getting bigger every day so only naive researcher will not take it seriously these days during data gathering and investgations.

Here are some Facebook big numbers from official Facebook Reports:

  • 1,49 bilion of MAU (monthly active users)

  • 4,5 biliton likes generated daily

  • 968 milion of DAU(daily active users)

  • 1,31 bilion mobile active users

  • 307 milion of people from Europe are on Facebook

  • Five new profiles are created every second

  • 300 milion per day of photo uploads

  • 20 minutes average time spent per Facebook visit

2. How to use Facebook in OSINT?

2.1 Download all feed from selected funpage.

How to run this code:

python funpage_feed.py -f FANPAGE_NAME -l 25

Parameters:

-f - Fanpage name

-l - Number of gathered elements each API call

Output:

FANPAGE_NAME_feed_data.csv

File Header:

"item_id","item_message","item_link_name","item_type","status_link","created_time","num_likes","num_comments","num_shares"

Comment:

This script could be very valuable in many situations. We want to know what posts were the most popular. We can also scrape feed from few similar fanpage and compare them.

2.2 Find out what fanpages are liked by another fanpage.

How to run this code:

python funpage_likes.py -f FANPAGE_NAME -l 1

Parameters:

-f - Fanpage name

-l - Number of gathered likes each API call

Output:

FANPAGE_NAME_likes.csv

File Header:

"fan_id","fan_category","fan_name"

Comment:

This script will allow us easily find out what fanpages are liked by particular fanpage and going further this way find out the relations between fanpages.

2.3 Search Facebook for specific phrase.

How to run this code:

python search_fb.py -q dev -t page -l 5

Parameters:

-q - Query to search

-t - Type of the results, default "page"

-l - Number of returned results

Output:

Output printed in console

3. Next steps

The aim of this article was to show and present the possibilities of using Facebook as a source of data, I described why Facebook can be so important in OSINT and how to gathenring data from it.

Please remember that this is just an introduction and covers only data gathering, you should realize that this is just a first step into OSINT world, in next we should know how to analyze collected data and how to get information from such data. These steps I would like to describe in Advanced OSINT series, so please leave the comment below if you are interested in such series of articles.

4. Additional information

The source code attached to this article is available on GitHub.

To start working with Facebook API you will need register your app and gain a developer key, you can do this at Facebook Developer Page. In above examples we used a Facebook Python SDK.

Basic - OSINT - M03 - Gathering data from YouTube

This post is a part of the OSINT series.

1. Why would you want to use YouTube in OSINT?

YouTybe is the most popular platform for sharing videos online today. What could be interesting is that Google has acquired YouTube in 2006 for about 1,65 bilion dollars (cash and stocks). Every researcher should noticed that this service is very valueable for doing some reconnaissance, it allows to view videos from all over the world, content here is usually released faster then in television and mainly with geolocation data included.

Here are some YouTube big numbers from official YouTube Press Room:

  • YouTube has over a billion users — almost one-third of all people on the Internet — and every day people watch hundreds of millions of hours on YouTube and generate billions of views.

  • YouTube overall, and even YouTube on mobile alone, reaches more 18-34 and 18-49 year-olds than any cable network in the U.S.

  • The number of hours people spend watching videos (aka watch time) on YouTube is up 60% y/y, the fastest growth we've seen in 2 years.

  • The number of people watching YouTube per day is up 40% y/y since March 2014.

  • The number of users coming to YouTube who start at the YouTube homepage, similar to the way they might turn on their TV, is up more than 3x y/y.

  • 80% of YouTube's views are from outside of the U.S.

  • YouTube has launched local versions in more than 70 countries.

  • You can navigate YouTube in a total of 76 different languages (covering 95% of the Internet population).

YouTube also provides a YouTube API which can be very usefull for data mining and gathering interesting information about video content. In this article we are going to use Python-YouTube-Samples written in Python language.

2. How to use YouTube in OSINT?

2.1 Search for video with keywords.

How to run this code:

python movie_keywords.py --q test --max-results 10

Parameters:

--q - Query to search

--max-results - Number of results to return

Output:

Output printed in console

Comment:

The above script will let you search YouTube for a specific phrase and get the movie results.

2.2 Search for video with keywords and geolocation.

How to run this code:

python geolocation_keywords.py --q test --location "52.23,21.01" --location-radius 5km --max-results 10

Parameters:

--q - Query to search

--location - Coordinates of the place

--location-radius - Radius

--max-results - Number of results to return

Output:

Output printed in console

Comment:

The above script will let you search YouTube for specific content like: movies, playlists etc which was uploaded at the specified location.

3. Next steps

The aim of this article was to show and present the possibilities of using YouTube as a source of data, I described why YouTube can be so important in OSINT and how to gathenring data from it.

Please remember that this is just an introduction and covers only data gathering, you should realize that this is just a first step into OSINT world, in next we should know how to analyze collected data and how to get information from such data. These steps I would like to describe in Advanced OSINT series, so please leave the comment below if you are interested in such series of articles.

4. Additional information

The source code attached to this article is available on GitHub.

To start working with YouTube API you will need register your app and gain a developer key, you can do this at Google Developer Console.

Basic - OSINT - M02 - Gathering data from Instagram

This post is a part of the OSINT series.

1. Why would you want to use Instagram in OSINT?

Instagram is a great platform for sharing photos and short movies where users have a possibility to using tags for their content, since 2012 Instagram is a part of Facebook which means that this platform is really important in social media world. Below some of Instagram numbers:

  • Monthly active users: 400 milion

  • Daily Instagram users: 75 milion

  • Percent of Internet users that use Instagram: 20%

  • Percentage of Instagram users that are younger than 35: 90%

  • Percentage of Instagram users that are from outside the USA: 70%

  • Number of active Instagram users in the UK: 14 milion

  • Number of shared photos: 30 bilion

  • Percentage of Instagram users that log in at least once a day: 49%

  • Average number of Instagram photos posted daily: 70 milion

Instagram also provides a Instagram API which can be very usefull for data mining and gathering interesting information about Instagram users. In this article we are going to use Python-Instagram library which will help us to use the API with Python language.

2. How to use Instagram in OSINT?

2.1 Discover followers for Instagram user.

How to run this code:

python user_insta_followers.py -u INSTA_USER_NAME

Parameters:

-u - Instagram user name

Output:

insta_followers_of_INSTA_USER_NAME.csv

File Header:

"follower_id","name"

Comment:

When having these kind of data about user we then will be able to find out more information about him, For example:

  • who are users whom user is following?
  • what kind of people they are?
  • do they might have any others mutual connections?

2.2 Discover friends for Instagram user.

How to run this code:

python user_insta_friends.py -u INSTA_USER_NAME

Parameters:

-u - Instagram user name

Output:

insta_friends_of_INSTA_USER_NAME.csv

File Header:

"friend_id","name"

Comment:

When having these kind of data about user we then will be able to find out more information about him, For example:

  • who are users friends?
  • what kind of people they are?
  • do they might have any others mutual connections?

2.3 Get photos of Instagram user.

How to run this code:

python user_insta_photos.py -u INSTA_USER_NAME -s yes

Parameters:

-u - Instagram user name

-s - yes/no if we want to download all user photos

Output:

insta_followers_of_INSTA_USER_NAME.csv

user photos as *.jpg files

File Header:

"photo_id","photo_url","like_count"

Comment:

Having such big amount of photos we can of course use those for:

  • finding faces on photos,
  • trying to recognize concrete places,
  • analyze photos and search for some kind of metadata (exif)

3. Next steps

The aim of this article was to show and present the possibilities of using Instagram as a source of data, I described why Instagram can be so important in OSINT and how to gathenring data from it.

Please remember that this is just an introduction and covers only data gathering, you should realize that this is just a first step into OSINT world, in next we should know how to analyze collected data and how to get information from such data. These steps I would like to describe in Advanced OSINT series, so please leave the comment below if you are interested in such series of articles.

4. Additional information

The source code attached to this article is available on GitHub.

To start working with Instagram API you will need to register as a developer to gain developer keys, you can of course find on Google how to do this or just simple read the instructions from Instagram Developer Site and enjoy your work. You can always use this helpfull Python script to gain developer keys.