New study shows that phishing simulations are not effective in training users

An unprecedented study has shown that embedded phishing training in simulations conducted by organizations does not work. Yet crowd-sourcing phishing detection is.

Image: Shutterstock/CalypsoArt

It is easy to compromise the network of a company by targeting employees through phishing attacks. They are the Your network environment’s weakest link.

Phishing simulations, also known as phishing tests, are becoming more common in corporate organizations. These simulations simulate real phishing emails landing in employees’ mailboxes. They don’t contain any malicious payload. They create a realistic phishing page. Then, they collect statistics such as who clicked and whether or not they gave credentials. How many users reported it the security staff, etc.

Professional phishing simulations can be used by companies, or they can create their own simulation with tools such as GoPhish.

Whatever the method, the goal is to get to know the employees better and increase awareness of this serious threat.

SEE: Ten tips to combat social media phishing attacks (free PDF). (TechRepublic)

Study of phishing on a large scale for 15 months

A recent studyThe computer science department at the University of Michigan published a paper on the subject. ETH ZurichThe university is a Swiss public university that focuses on engineering, science and technology. The study was 15 months long and involved large organizations (more than 56,000 employees, approximately 14,000 of whom were targeted by the study). It is the longest published study in terms both of length and scale.

The email sent phishing emails to lead to a page phishing, or contained malicious files that encouraged the user to launch a dangerous action, such as enabling macros or providing credentials.

Phishing emails can contain either brief or longer warnings (Figure AHowever, other emails didn’t contain any warning whatsoever.

Figure A


Two warnings in fake phishing emails are: short and long

Source: ETH Zurich Dept of Computer Science

A reporting button was also available in the email client that allows employees to report any phishing attempts. This button was advertised in company news prior to the study.

The simulation can take the user to an educational page that explains what happened, how to avoid phishing and what to do next. A second instructional video, additional quizzes and learning materials on phishing were also available, but users weren’t required to view it or even read it. Some users didn’t get that educational page.

SEE: Phishing attacks at work are more common in digital natives than among their Gen X, Boomer and Gen X colleagues (TechRepublic)

Which users fell for phishing more often?

The study examined the dangers of dangerous computer use, as well as gender and age.Figure B).

Figure B


Divided by different demographics, percentage of dangerous actions taken out of all phishing email sent

Source: ETH Zurich Dept of Computer Science

Computer usage

Employees who have a more specific use of computers (e.g. branch workers who use one software exclusively) click on more phishing links to perform more dangerous actions that the rest of the users.

Age range

The youngest employees were more likely to click on potentially dangerous links than those who were older. Phishing was more common among employees aged 50-59.


The study revealed that computer use combined with gender was significantly more important than gender.

SEE: Shadow IT policy (TechRepublic Premium)

Phishing for long periods

The study lasted 15 months and found that only a few employees fall for phishing multiple time, especially among the youngest.

It also showed that many employees are susceptible to phishing if exposed to it for too long. Researchers at ETH said that phishing will affect a large percentage of employees if they are exposed to phishing emails over a prolonged period of time.

Warnings are useful, educational pages are not

Although the warnings contained in the phishing emails were effective in preventing users clicking on the links, they weren’t as effective as short warnings.

It was even more surprising that users who were phished clicked more often on phishing sites after they had received the educational page. Researchers emphasized that this result could only be used to deliver voluntary training. Other methods may provide different results.

This significant finding was discovered by researchers using the post-experiment questionnaire completed by employees. The possible reason could be a false sense that the deployed training method provides security: 43% chose the option “Seeing the training website made me feel secure” while 40% selected “The company protects me from spam emails”. Future research will be needed to determine if this is due to misunderstanding of the training web page (e.g. employees believed they were protected against a real phishing attack) or overconfidence in the IT department.

SEE: Phishing attacks – A guide for IT professionals (free PDF). (TechRepublic)

Employers are an asset in fighting phishing

According to the study users continued reporting phishing emails over time, and that there wasn’t any “reporting fatigue” in the company. It was clear that a significant percentage of users were actively involved in reporting. Reporters who had the highest expected computer skills were most active. Positive feedback was also a motivator for reporting users.

Within five minutes of receiving the email, 10% of users sent the reports. Between 30 and 40% of reports were sent by users within 30 minutes.Figure C).

Figure C


Source: ETH Zurich Dept of Computer Science

However, for such crowd-sourcing methods to be successful, employees need an easy and convenient method to report phishing cases. It seems that a button in the email client is a good option.

Disclosure: Trend Micro is my employer, but these views are mine.

Also see