Identifying Creators of Disinformation
We all know the saying “where there’s smoke, there’s fire”, which means that if there is telltale evidence of some event, the event is probably occurring. But when confronted with information online, often times where there is smoke, there is mostly just smoke. When confronted with an outrageous, bizarre, or explosive claim posted on social media that is getting traction, don’t assume there is a fire, go look for it. This article goes into detail about how to identify creators of disinformation. Look at the account, the audience (following and spreaders) and the message(s).
First establish if the account can be traced to a person or organization to do this:
- Twitter’s blue checkmarks indicate that a user is verified and therefore less likely to be automated or disingenuous. However, there are some obstacles to the authenticity of this check, especially since Twitter Blue is now an opt-in monthly subscription. If an account has a blue checkmark but a weird handle tied to a prominent person this is still an indication of inauthenticity.
- Look if the person has a full name.
- Check their profile for anything that leads to further information about the person or organization. Further information may suggest that this is a genuine account.
- Is there a website that leads to a person with the same name?
- Did the profile change dramatically over time? (Use Google Cache to check this)
- Using an auto generated name will provide an odd name so check to see if it is odd.
Check the Profile
- Check if the account name or the picture have already been used on the internet. Search for the name in all variations to look for accounts consistent with the name. Use usersearch.org or social-searcher.com to check if the other accounts correspond with the profile researched.
- If you cannot find any name associated with the account, try to form an image of the account, from indications of geographic location, hobbies, or political leaning. From these clues, search for other combinations of the name that include these interests, e.g. if it was first John, try John NFL.
- Use a reverse image tool to check if the image has also been uploaded to other locations. We can check if the details in the image in the other locations match the original account or if they have discrepancies. For example, someone named Jane being named Sarah elsewhere, if this is the case, we have found a suspicious account.
- Look at the creation data of the account using tools like Social Bearing or Tweet Reach. For Facebook, search manually for the first posts if possible. If the account is old there could be traces leading to the person behind the account.
Check the ID
- Another data point we should collect is the user ID. Every user on every platform has a personal unique ID for the platform they use. Search the ID with a tool, for example for Twitter you can use https://tweeterid.com/
- In Facebook the ID will often appear in the URL of the user. If not then use a Facebook ID finder to get the ID: https://lookup-id.com/
- This ID is important if an account switched names, because even though an account might change the name or handle of his account, the user ID will always remain the same.
- This step is useful to check if the account has changed its name. Name changes can indicate if accounts have been hijacked or if a disinformation account has changed it’s agenda.
- On Facebook we should also always check if the username in the URL (though it might be a nickname) is consistent with the name on the profile. If this is not the case check if there is any discrepancy, e.g. a Turkish male name in the URL but a female Latin-American woman in the profile, we have an indication the account might be inauthentic.
Checking Suspicious Content
A clear indication that someone is attempting to spread disinformation, is that their content is inflammatory or false. You can also check if the content contains evidence that it is consistent with a wider plan to spread false information using multiple fake accounts.
Check the Content
- The use of hashtags can give away a disinformation purveyor. They will jump on different hashtags to gain traction for their messaging, frequently polarizing or inflammatory hashtags.
- Check if the account uses original hashtags or if they jump on existing hashtags. Check if the hashtag is on other social media sites using: https://www.hashatit.com/. Then try to find the first user using the hashtag to see where it originated.
- Use InVid reverse image tools to analyze the media present in their posts. Look for obvious inconsistencies in the posts that reveal that the post is fake.
- Check if the content is original, disinformation campaigns will sometimes steal media like video and images and reuse it. If they insist that an old video is new that is clear evidence of disinformation.
- Reverse search the text using quotation marks to see if there are already posts with the same text.
- Look at the text for errors that a native English speaker would not make. A native speaker would find errors in the text and a disinformation spreader may not be a native speaker.
Check Interaction with Other Users Around Content
- Check if the account is tagged with other accounts and if these tagged accounts are high profile accounts. If they tagged high profile accounts, they may be trying to spread their content as far as possible.
- Not engaging with users except for sending messages might be a sign of a paid troll. Low ranking trolls will often not engage with users.
- If there is a link in a post, then you should reverse search the link. If the post is suspicious and shared by suspicious accounts, then look for similarities between the accounts as they may be part of a network of accounts.
- Influence operations may have websites attached, check if there are strange websites repeatedly featured in the content and check the background of the websites and the accounts sharing them.
- Look at how fast a tweet or account spreads using Hoaxy: https://hoaxy.iuni.iu.edu/. Look at strange accounts within the retweets or shares and likes.
Look at the Purpose of the Post
- We should also try to understand the intention of the post. They could be simply attempting to gain attention for a nonpolitical claim or clout chasing. There might be signs of trying to influence public opinion through polarizing messages. They might be trying to intimidate or harass someone. Examining the content of the post can give insight into what the user is trying to achieve.
- Check if the messaging of the user changes over time, or if their messages are playing into the narrative of an ongoing disinformation campaign or geopolitical camp.
Checking a Suspicious Audience
It is necessary to examine the audience of a suspected spreader of disinformation for evidence that many of their followers are automated or inauthentic accounts.
Check the Number and Reshares or Followers
- A new account or a relatively unpopular account that gets a large number of likes/shares quickly is suspect. Look at the account followers, if there are high-profile accounts who have shared the post, this may explain the large number of shares for their content. A post from a small account that is shared by a high-profile account which clearly has biased views is suspect.
- Check the first followers on Twitter. If the same batch of accounts are the first followers for each other, then this is suspect.
- Look at the ratio of followers to accounts it follows. An account with a huge number of followers rapidly accumulated and very few followed accounts is suspicious.
- Check if there is an organic spike in followers by using socialblade.com long straight spikes up or down are suspicious. It either means they went viral or had a fake following composed of fake accounts. Check if the post has any popular posts around the time it got many followers. A negative spike could mean a mass deletion of fake accounts.
Check the Audience
- Determine if the group of followers is credible. If the content doesn’t belong to a high-profile person or organization and they are relatively new and have a huge following (over 1000 in a month) then this is suspect. Check for anonymous accounts among their followers and see if they are the majority of the group. Check for similarities in the online biographies of the followers, such as similar creation date and similar writing in the biographies. To check quickly if two Twitter accounts follow each other use: tweetbeaver.com/mutualfollow.php
