The Vermont non-hack: what went wrong, what went right

 

It was the last day before New Year’s weekend at Vermont’s Burlington Electric Department, just a few hundred yards from icy Lake Champlain.

An employee checked personal e-mail on a company laptop, like many workers at many companies around the country.

But on this morning, the Yahoo e-mail connection set off a warning that would be heard not just at Burlington Electric, but around the world.

Soon, it made headlines in the Washington Post as a Russian hack of the U.S. electric grid, triggering fears that this small, progressive utility near the Adirondacks was an international cyber target, possibly the first step of a foreign cyber invasion into America’s critical infrastructure.

The truth was far from it, according to Burlington Electric.

“We’ve been trying to—ever since the Post inaccurately put the story out—we’ve been trying to put that genie back in the bottle,” General Manager Neale Lunderville told Archer News.

In the fallout, questions remain—what went wrong, what went right, and will this incident make some utilities shy away from sharing potentially crucial cybersecurity threat information with the government, for fear it will be twisted—like the Vermont grid non-hack—into something bigger and more destructive?

 

Burlington Electric tweeted that there was no indication that the grid was compromised.

 

How it happened

The non-hack started more than a week ago—Thursday, December 29, according to Lunderville.

Federal officials sent out a special report on GRIZZLY STEPPE, a name for Russian civilian and military intelligence services cyber attacks.

The Department of Homeland Security and Federal Bureau of Investigation Joint Analysis Report included a list of signs to look for, called indicators of compromise, that could potentially be associated with Russian hackers.

The report recommended that network administrators check their systems for these possible clues, which may or may not indicate malicious activity.

“The idea was to share information because information-sharing is essential, we think, to the protection of critical infrastructure and to overall cybersecurity for the country,” a DHS official told Archer News by phone.

Burlington Electric updated its systems with the information, and began to scan Thursday night, according to Lunderville.

 

Illustration from the DHS & FBI Joint Analysis Report on Grizzly Steppe.

 

New Year’s Eve Eve

The next morning, December 30, a worker signed onto Yahoo e-mail.

“That computer starred communicating with an IP [Internet Protocol] address that was listed in the indicators of compromise,” recounted Lunderville. “Our system flagged that and sent us a warning.”

“Once we detected that traffic we immediately removed that computer from the network and isolated it and contacted federal authorities,” he added.

Business computer

However, a laptop communicating with a possibly suspicious IP address does not automatically mean the U.S. electric grid was hacked.

The computer was for business, not for operating the grid, Burlington Electric said.

“The affected laptop was not connected to any computer system with access to, or control of, the grid,” explained Christopher Recchia, commissioner of Commissioner of Vermont’s Public Service Department.

“There’s no indication that any of our systems were compromised, nor is there any indication that any customer data was impacted,” said Lunderville.

 

The Burlington Electric Department serves 20,000 customers in and around Vermont’s largest city.

 

What was it?

The investigation continues into just what the Burlington Electric system found, but Lunderville said it was not unique to his organization or even to the utility sector as a whole.

“The truth is, there a lot of potential cyber threats that are swirling around on the Internet. Some of them are the more garden-variety and some of them are more sophisticated,” he said.

“As utilities, we take this cybersecurity very seriously and we work very hard and make sure that we are scanning for all threats. regardless of their severity, and when required, reporting those to federal authorities,” Lunderville said.

 

Burlington Electric responded to reports of Russian hacking on its Facebook page and on its website.

 

Russian hackers?

No compromise, no customer info stolen, no grid attack, according to Burlington Electric.

But within hours, The Washington Post reported that Russians had hacked the 20,000-customer Vermont utility and penetrated the grid.

Suddenly, as the last hours of 2016 wound to a close, Lunderville and his team found themselves in the center of a maelstrom, trying to set the record straight on what looked to the world like an international incident with serious consequence.

The team thought they had done the right thing—scanning, finding, reporting.

“All that worked,” said Lunderville. “What didn’t work was someone inside the federal government took the information, misinterpreted it, leaked it to The Washington Post, and The Washington Post ran the story without confirming it first.”

“The last two things are the root cause of the issue, not the cyber issue, the misinformation issue,” he added.

 

A Washington Post headline incorrectly stated that Russia had hacked a Vermont utility, according to Burlington Electric. The Post later wrote a follow-up story about the incident.

 

Root cause

But some analysts say the problem is the GRIZZLY STEPPE report itself.

“Burlington Electric did everything they should have,” Robert M. Lee, CEO of cybersecurity company Dragos, Inc. told Archer News. “The problem was that the DHS report should have never gone out in that state.”

The GRIZZLY STEPPE report was “riddled with errors” and provided information that in some cases was so non-descriptive and without context that it was “nearly useless,” Lee wrote in blog posts.

“It failed to meet its own objectives in every category and simply was not good technical data for use by a large audience,” Lee said.

For example, bad guys may use an IP address for malicious purposes for a while, then abandon it, so simply listing suspicious IP addresses may not give a defender enough information, he said.

And using that limited information to blame another country, as some federal officials apparently did?

“To start making claims of attribution to a state such as Russia just because some poorly constructed indicators alerted on a single laptop is dangerous,” Lee wrote.

 

 

The header of the GRIZZLY STEPPE Joint Analysis Report.

 

“John Smith”

This lack of information could make security even harder, said Ron Fabela, senior threat hunter with Dragos.

“Imagine if a Russian operative by the name of ‘John Smith’ was trying to gain entry to the U.S.,” Fabela said to Archer News. “This person was dangerous, so they put ‘John Smith’ on the no-fly list.”

“Is this an effective security control?” he asked. “Without additional context, stopping every John Smith from boarding a plane is actually counter-productive to overall security.”

“This is what happened with the Vermont ‘hack,’” he said. “Someone in the government put a lot of generic identifiers on a list of ‘bad things.’ The utility, using but not verifying the identifiers given to them, flagged a connection to Yahoo and reported it to DHS as a bad thing. The current political environment and leak to the press caused a spiral of misinformation.”

 

Map showing Burlington Electric near Lake Champlain. Image via Google Maps.

 

Lasting damage?

After eight intense days—straight through the holidays—Burlington Electric is finally wrapping up its triage.

But some analysts say the incident will cause lasting damage, as utilities see what can happen if they share cybersecurity information with federal agencies.

Some utilities may not have the resources to research and validate the indicators of compromise provided by DHS and may choose to ignore future reports, according to Fabela.

“For other asset owners in the industry it’s equally grim,” he said. “If we can’t validate, then ignore. If we must execute, don’t report. If we must report, then plan for a potential leak to the press.”

Without trust, with less reporting, the U.S. electric grid could become less safe, some worry.

 

Cybersecurity information included in the GRIZZLY STEPPE Joint Analysis Report.

 

DHS response

Archer News contacted DHS for comment.

A DHS official who declined to be identified said he could talk about the GRIZZLY STEPPE report, but could not address specific criticisms.

“The Joint Analysis Report is obviously supposed to provide information that will help infrastructure owners protect themselves, or at least identify if there’s any malicious traffic within their regular traffic,” he said. “We did provide actionable and detailed tactics and techniques and procedures that these actors use that they can leverage when they detect these types of intrusions into these networks.”

The official said federal agencies are required to protect the identity of their customers in this kind of information-sharing set-up.

 

The GRIZZLY STEPPE report provided some alternate names for Russian military & civilian intelligence services.

 

Lessons learned

People watching over computer networks of America’s critical infrastructure must take a closer look at this kind of intelligence information, analysts say.

“Network defenders must always be suspect of the data they receive externally even from trusted sources,” said Lee. “That does not mean it should not be used but that it should be validated, enriched and understood before using it.”

Another lesson—anyone looking at the information—from the federal government to the media to the public—may need to move more cautiously.

“Nobody who looked it and knew anything about this type of potential threat through it warranted hitting the ‘red alert’ button, and the leaker and The Washington Post did,” said Lunderville.

“I think one of the lessons is this is not necessarily intuitive information,” he said. “We’re dealing with complex networks, complex technological issues and complex threats, all of which need to be reviewed very carefully before drawing a conclusion.”

 

Burlington city hall. Burlington Electric is municipally-owned. Photo: Doug Kerr from Albany, NY CC BY-SA 2.0

 

No more sharing?

Will utilities stop sharing cybersecurity information with the government?

“If I share my information, my New Year’s going to get ruined?” joked Lunderville.

“I think that it would be fair to draw that conclusion, to view it that way,” he said. “But I would caution anyone from drawing that conclusion.”

Why?

“Overall, our federal partners have been great to work with,” Lunderville explained. “Just because one or two people decided to misinterpret the information and use it for their own ends, whatever those ends may be, political or otherwise, we should not say say suddenly, ‘We’re not going to work with the feds.’”

And, he believes, the non-hack will spur change at the federal level.

“The next time this kind of information comes up, folks will stop and understand it before they react to it and do something stupid with it,” Lunderville said. “The utilities shouldn’t stop sharing that information while we wait for that to happen.”

Will other utilities share this same positive attitude? It is too early in the new year to tell. But many will be watching to see if the Vermont non-hack will break trust and put a freeze on cybersecurity information-sharing in 2017.

 

Featured image: Burlington, Vermont. Image via Pixabay.