To Talk of Many Things: Of Vandals, and Cuts, and Cables, and Pings
The time has come,’ the Walrus said,
To talk of many things:
Of shoes — and ships — and sealing-wax —
Of cabbages — and kings —
And why the sea is boiling hot —
And whether pigs have wings.’
(Excerpt, Lewis Carroll’s The Walrus and the Carpenter)
Here’s an open information security topic worth examining more closely: the recent vandalization of yet another fiber optic cable on the west coast.
A total of eleven cuts have been made since last July on fiber optic cables in the greater San Francisco/Oakland area. The most recent cut occurred on June 30th. The FBI had already asked the public for help with information about the first ten cuts, made in these general locations at the time and date indicated here:
1) July 6, 2014, 9:44 p.m. near 7th St. and Grayson St. in Berkeley
2) July 6, 2014, 11:39 p.m. near Niles Canyon Blvd. and Mission Blvd. in Fremont
3) July 7, 2014, 12:24 a.m. near Jones Road and Iron Horse Trail in Walnut Creek
4) July 7, 2014, 12:51 a.m. near Niles Canyon Blvd. and Alameda Creek in Fremont
5) July 7, 2014, 2:13 a.m. near Stockton Ave. and University Ave. in San Jose
__________
6) February 24, 2015, 11:30 p.m. near Niles Canyon Blvd. and Mission Blvd. in Fremont
7) February 24, 2015 11:30 p.m. near Niles Canyon Blvd. and Alameda Creek in Fremont
__________
8) June 8, 2015, 11:00 p.m. near Danville Blvd. and Rudgear Road in Alamo
9) June 8, 2015, 11:40 p.m. near Overacker Ave and Mowry Ave in Fremont
__________
10) June 9, 2015, 1:38 p.m. near Jones Road and Parkside Dr. in Walnut Creek
The FBI presented these first ten cuts as a single, undivided list. After looking at the dates and times, one can see these cuts may have occurred not as discrete events, but as three separate clusters of cuts. The first cluster occurred within a five-hour span; the second occurred nearly simultaneously at two points; and the third cluster occurred within three hours. The three clusters took place after dark, during the same evening. The tenth cut may be a one-off, or it may be connected to the third cluster as it took place within 14 hours of the eighth and ninth cuts.
The most recent cable cut, occurring this week, did not fit a pattern like the previous ten cuts. Reports indicate the cut was near Livemore — a new location much farther to the south and east in comparison, and only one cut reported rather than two or more.
Is this latest cut an outlier, or were perpetrators interrupted before they could cut again?
Taking a closer look at the previous cut events, we can see there must have been more than one individual involved in the cuts, and they may have been coordinated.
Cluster 1: The first cluster from one year ago, the evening of July 6-7, took place over a distance of roughly 34 miles. Cuts 1 and 2 are nearly 30 miles apart; by private car they are more than 40 minutes drive or more than an hour and a half apart by public transportation. At two hours between events it’s possible the same single perpetrator made these cuts, but only if they traveled by private car and if they knew exactly where to go and what to cut.
Cuts 2 and 3 are also about 30 miles apart, in the opposite direction. It would be nearly impossible for the same single perpetrator to make these two cuts back-to-back since the time window between the cuts is only 55 minutes. Could a single person make it up out of a manhole from one cut, into a vehicle, drive nearly 40 minutes, park, open and climb into another manhole, then cut a fiber optic cable?
Cuts 3 and 4 were not made by the same single perpetrator. They are roughly 34 miles apart, and the time window between the cuts is less than 30 minutes. To cut-exit-manhole-drive-park-enter-manhole-cut would require traveling at speed that would surely draw attention, even at 12:30-ish in the morning.
Cuts 4 and 5 were made one hour and 24 minutes apart, and the sites are about 20 miles apart. This last cut was the farthest south of the first cluster.
Cluster 2: The second cluster from February this year, consisting of only two cuts happening at what appears to be the same time, suggests there was more than one perpetrator involved. The two cuts occurred at the same time, but within 0.2 miles apart — as if two persons within line of sight cut at the same time. The cuts also occurred near or at two of the previous locations from the first cluster.
Cluster 3: Cuts 8 and 9 occurred 40 minutes apart, yet the sites are roughly 30 miles apart — too far once again for a single perpetrator. Both happened within the hour before midnight local time.
The tenth cut may have been related to third cluster, as noted previously — but it broke from the established pattern. The first nine cuts all occurred after 9:00 p.m. but before 3:00 a.m. local time. The tenth occurred at 1:38 p.m., in broad daylight.
Cut 11, the most recent on June 30th shared this same attribute. It happened some time between 4:20 a.m. and 7:45 a.m. local time (14:45 UTC), near Livermore, CA, to the east of the previous ten cuts.
Were the same so-called vandals at work for all eleven cuts? If so, were they getting cocky, having not been caught on nine earlier occasions?
Or were they getting desperate?
The implication, assuming desperation, is that these were not the acts of vandals, but a focused effort dedicated to network disruption?
Or perhaps not disruption with intent to halt or disturb, but disruption to map network response and content movement?
What might these vandals have been looking for, as they cut at one end of an area across the bay from Palo Alto and Mountainview across to an area east of Silicon Valley?
Did they finally learn something when Microsoft issued a formal status notice regarding disruption of its Azure cloud services — perhaps which fiber served the Azure data farm?
6/30
Network Infrastructure – West US, South Central US – AdvisoryFrom approximately 14:45 UTC to 21:45 on 30 Jun, 2015 UTC customers may have experienced intermittent connectivity issues to Azure services deployed in West US and South Central US. Root cause for this issue is attributed to a fiber cuts in the Western US Region. This incident has now been mitigated.
Or were they looking for fiber optics serving the Lawrence Livermore National Laboratory, home to other data farms and a number of sensitive research projects?
Or were they looking for the fiber running out of San Francisco, serving headquarters of businesses headquartered in the city like Wells Fargo?
UPDATE – 5:10 PM EDT – Here’s the graphic as promised, mapping the approximate location of cuts per the FBI’s list. The 11th cut is arbitrarily parked near Livermore as more specific site information was not provided. Cuts are labeled in chronological order.
![[graphic: via Google Maps]](https://www.emptywheel.net/wp-content/uploads/2015/07/BayArea_FiberCuts_03JUL2015.png)
[graphic: via Google Maps]
Not that these cuts may be related to the undersea cuts from a few years ago, but since your talking espionage, I wonder what provider(s) these cables belong to and more specifically, what company will repair the cuts. Wouldn’t it be a perfect opportunity to hack ?
LAT reported “At least two companies acknowledged that fiber-optic cables they own were compromised in the attack” — another outlet labeled these as “(internet) backbone providers.” I expect that either these firms do their own cable work, or they have a contractor they work with closely. If they are close, or if there are moles inside the backbone providers, they have plenty of opportunity to do spying within the course of regular daily work. I don’t see the cuts as opportunities to insert intelligence collection later.
I have a theory this was an attempt to locate a specific line, one that might even be dark fiber.
I also find it interesting that CalTrans doesn’t have traffic cams near any of these cuts.
interesting .
fiberoptic cable used for what?
what tool(s) would you need to cut such a cable? a battery operated reciprocating or cicular saw, a propane torch, a hacksaw, etc.
it seems the cutter knew what to look for (a fiberoptic cable) – as opposed to electrical or old telephone or traffic signaling or street car or gas line (whoopee).
were the cuts entirely thru the cable?
i vote for anthrax-stye military operation designed to stir public concern re cibersec.
Heh. You’re funny. There are websites for businesses dedicated to cable cutting tools. Might even find a specialized tool on Amazon. I’d be more interested in knowing how big the cut cables were — one-person or a multi-person job?
i can buy fibreoptic cables online, eh – and i thought i owned most of the tools pubicy available to americans :)
i’m accustomed to cutting service entrance cable and rebar, but a (presumably) large diameter fiberoptic cable seems like another mountain higher.
i’d think you’d have to know just what you were doing to have the right tools and go to the right places.
Interesting and to bad the place I grow up is in the news for this, Niles, Calif.
Most of the fiber that runs near my house about 1-1/2 inches in dia. but is buried underground in a plastic conduit about 3in in dia. Sprint or who ever does have fiber vault near me, no humans but lots of cameras and AC. Then 10ft high fence with barb wire.
I come from the trades so to me a battery saws all would be the fastest way since you don’t care about how clean it is.
Looking forward to see what else you find Rayne and thanks for the Cal Trans link
Wish I knew what the cables’ cuts looked like — might be why the word “vandals” is used so frequently. If the cuts were fast and messy, it’d be assumed somebody was simply cutting service and not looking to obtain content.
But…cutting the cables does actually provide intelligence. We know, for example, that the last cable cut did impact a major corporation’s data farm service.
On the other hand, “vandals” as a label is an easy redirect, a subtle means to tell average Joe, “Nothing to see here (just meddlesome kids), move along.”
I think these guys *knew* they weren’t cutting a major telecom provider’s lines. The locations suggest they were looking at services in transportation right-of-ways, though, just not near CalTrans cameras.
Just skimmed thru, so I don’t know if you’ve asked this, but my question is: who could have gotten the info as to where these specific cables were? Somehow I doubt there were huge billboards next the roads saying “Hey! Look Important Fiber Optic Cables, Here!” with a blinking neon arrow sign pointing down
Good question, which I’m scratching my head about. Somebody knew where these vaults were, but not specific cables? Or did they know which cables…
Is it possible they had a map about cable, but not a map showing network response when cables were severed?
IMO, some third party could get network mapping online from hackers, but they might not have access to how the network operates. I could be persuaded otherwise, though.
Did you ever walk the Secret Sidewalk, jo6pac?
It was one of my favorites . . . .
LOL yes many times;)
This reminds me of the oil transport companies who fought disclosure of their tanker train routes to the public (for safety preparation purposes) on the ground that ‘terrorists’, knowing the routes, might intercept said trains. How do today’s ‘vandals’ know what cable is where? I suppose anyone with an in at Verizon FIOS, for example, would know a lot. Of course, this cable infrastructure isn’t as critical as, for example, the water system, which we have known for decades is vulnerable to anyone injecting a toxin at the nearest hydrant — they never did address that infirmity, did they?
“this cable infrastructure isn’t as critical…” Um…except that some of this cable infrastructure carries content as valuable or more so than any railway or waterline.
I should point out that Kaiser Permanente, a managed care provider with ~10 million subscribers, has its headquarters in Alameda County where a number of these cable cuts occurred. Port of Oakland is also located in that county — it’s the US’ fifth busiest container port. How does the port’s shipping data travel in and out?
If you were trying to find data from the Lawrence Livermore National Laboratory those would be good places to try to tap. But they just cut the lines we were told . . . .
Seems odd they didn’t try closer to Livermore from the beginning, if that was the target. The first cut was west of San Francisco, +30 miles north of Livermore. Seems too far off.
I did check for any submarine cable landings in SF/Oakland area — didn’t find any on the map. ~shrug~
I’m not sure what goes where, fiber-optically, but Niles Canyon (Highway 84) is one way to go from Fremont to Eastern Alameda County. Eastern Alameda county is also home to CHEVRON!
There is the Vallecitos Nuclear Center that is a few hundred feet from and is visual from 84. It is owned by GE Hitachi Nuclear Energy.
84, for a large section, is a relatively remote and somewhat winding and dangerous road. Most people would be keeping their eyes on the road and not look for strange trucks. It seems that most of the actions were on this stretch of road (which begins at Niles/Mission) in Fremont. 84 starts near the western end of the Dumbarton bridge in East Palo Alto and ends in Livermore. Maybe the Stanford Band Pranksters did it.
#11 sure looks right next to the lab! The math that the lab does requires massive computational firepower. I am sure that much of the data is shared elsewhere (over a fiber network) for number crunching.
I would think that being too close to the lab or the nuke plant would be a little obvious.
The lab does much much more than just nuke stuff. They are really big on advanced explosives and the like.
I must be tired, because the first time i’d read that, I saw it as, “The first cut was 30 miles west of San Francisco …”
.
Of course that would mean the Farallons — and that could only mean one thing —- it’s the SHARKS!111! ;-)
There’s got to be lots and lots of people who know where the cables are – co-opt any of them and you are set. (At least the physical locations, if not what cable belongs to who.) All the cable repair people on all the repair trucks for all the providers, to start. And all the “infrastructure” people who work for the providers directly. And those “call before you dig” people. And I bet the municipalities also keep track of what all is buried on their property, too. The knowledge would be widespread because it’s only recently that fiber optic cable became hacker fodder – for a long time, it was just part of the basic stuff of life, like phone wires and sewer lines.
.
And my point about the municipal water supply lines is that if you tap into them, you can immediately poison and/or kill large numbers of people – I would certainly consider that “critical”. A very easy blackmail possibility. Is DHS there for us, on that front?
Wonder how long it takes to open the person hole cover, place a ladder and get the job done.
I hope these guys use cal osha guides for warning “Construction Ahead” , “$300 fine for speeding” signs a construction zone and proper traffic cone placement.
Pretty sure their late night cuts avoided concerns of appearance in re safety. It’s the last two cuts during twilight/daylight that intrigue me – were there any other cameras in the vicinity, since CalTrans cams don’t appear to be nearby?
I don’t know, I’m just a dumb guy on teh internets, but the timeline, to me, says professionals. It was coordinated by a group of people, with access to infrastructure maps, and with an eye towards weaknesses in technological security.
.
I guess another question is: what was their motivation?
.
Was it a “Jack Reacher” type scenario? Were they only after one? And the rest were smokescreens? Or, were they attacking several cables?
I’m working on more graphics showing the direction of the cuts during each cluster-event.
My current theory is that they were watching fail-over, to see which direction network traffic moved to ‘auto-heal’ and re-route in event of a break of some sort.
My second, weaker theory is that there is *something else* they are looking for in a trapezoidal area from Oakland to Sacramento (north end) and Fremont to Livermore (south end). I don’t think Silicon Valley is targeted per se, since only one of the eleven cuts happened toward the west of the bay in that area. The direction of the work suggests to me they know one end, but they are still locating another point in the network. I am wondering if they are looking for a cable route following a specific right-of-way.
Of course there is a rather large assumption made in nearly all reports so far, even my own above — that the cuts were made quickly and the point of damage cleared quickly by the perps. The pattern from the first and second cluster of cuts suggest the perps may have stuck around — did they insert something in the cut, and the cut only become evident when they exited the scene? ~shrug~
The pattern from the first cut cluster also suggests to me there were at least 2-3 perps, not 1-2. Why were that many needed?
As Woody Allen once said: “I was nowhere near Oakland!”
As far as cluster one is concerned, it is California, they were committing a crime, I think you can eliminate the only if they were using a private car qualifier, as if that is a big deal. Yes, they probably drove a car and did not take saws on BART.
thanks, rayne, for this fascinating post. when the cuts started getting press my spidey sense went off. thanks for teasing out some interesting possibilities.
(and Mountain View is two words)
Rayne,
“A total of eleven cuts have been made since last July on fiber optic cables in the greater San Francisco/Oakland area. The most recent cut occurred . The FBI had already asked the public for help with information about the first ten cuts, made in these general locations at the time and date indicated here”
I wonder if you could clear this up. Is there a word or punctuation missing?
Thanks.
Fixed – sentence now reads,
I’d meant to go back and insert that date after validating against MSFT’s outage notice, but got sidetracked by a crapload of maps I was running.
The LAT report on these infrastructure attacks is as interesting for its spin as it is for the information offered. They mention the Oregon attack and the earlier electrical substation attack but immediately and authoritatively project that they are not connected without having any people in custody. The use of the term ‘vandalism’ also deflects and seeks to sooth the reader who might otherwise begin to think that these attacks are organized sabotage.
These type of sabotage attacks are happening throughout the country but few are reported beyond the local news or even when they are, such as these repeated FO attacks in Cali, their significance is downplayed.
There are small and loosely connected, for security reasons, groups and individuals who are practicing ‘Decisive Ecological Warfare’ also called Monkeywrenching with the goal of bringing down Industrial Civilization so that a remainder of humanity will have a viable biosphere capable of sustaining human and other life on this planet.
Time Is Short!
I don’t buy these cluster of cuts as “monkeywrenching.” These particular efforts contribute nothing in alignment with the concept of “green resistance,” and they’d only deter their cause if they were looking to increase support for ecological issues.
Your description of “Decisive Ecological Warfare” as “bringing down Industrial Civilization” also doesn’t sync with “green resistance,” fitting more closely with outright anarchism. Were these clusters of cuts anarchist? Possibly, but if they are, they are merely poking a grizzly bear with a pencil — hardly on par with 1960s Weather Underground, by comparison.
I hope you continue to investigate these actions and possibly determine what larger effects they may cause. It seems to me they may be attempting to create a cascade failure to overload and bring down a much larger section of the system.
These Underground militant actions are not designed to attract people to support ‘issues’ they are a direct act of war/sabotage on our Industrial Civilization that requires ecocide for its continued existence.
There are aboveground organizations that offer many levels of commitment to those who want to participate in this struggle. I’m not referring to the Big Green orgs that are funded and controlled by corporations but groups such as Earth First and DGR who offer a clear analysis of the problems we face.
You’re making me laugh, really. Grizzly bear poked with a number 2 Ticonderoga school bus yellow pencil, that’s the scale of damage at best. Might have caused some monetary loss and a few hours of highly localized annoyance, but daily cyber attacks on ISPs and their customers cause far more damage than these cuts did if the perps’ aim was economic disruption.
The sad part of these cuts? The ISPs will only increase surveillance and increase their pricing as necessary. So much for “underground militants” helping liberate anybody or anything.
Land whales…
Rayne’s mention of the fiber optic line to ‘auto-heal’ reminded me of former Qwest CEO Nacchio excitedly announcing Sonnet rings around cities. In 1997, the following was written about Qwest and its plans to use railroad rights of way to quickly lay fiber in the ground.
“The California expansion will also include an additional SONET ring in the network. This self-healing system adds even more security and reliability to the network by allowing instantaneous rerouting in the event of a fiber cut. This architecture will virtually eliminate network downtime for Qwest Network customers. “We are extremely pleased with our progress to date and our ability to now expand the Qwest Network,” said Joseph P. Nacchio, president and CEO of Qwest. “With this expansion, the rapidly growing Southeastern and California markets will be able to take advantage of the Qwest Network, and we will be able to directly connect to the important cables serving Latin and South America.” The cost of the new network expansion will total approximately $375 Million. Qwest is also announcing four dark fiber contracts with major telecommunications providers totaling $89.4 Million. By supplying firms needing additional fiber capacity, Qwest is able to reduce the net expense of this expansion thus sustaining its low cost position. Qwest currently anticipates that funding for the balance of the cost will be provided by additional dark fiber sales and, if required, the incurrence of additional indebtedness. One major telecommunications service provider has already committed to acquire 24 dark fibers along the routes from Charlotte through Florida and back to Atlanta. This company will pay $70.6 million for the dark fiber acquisition. In addition, Qwest has signed three dark fiber customers for portions of the California central valley route whose aggregate payments to Qwest will be approximately $18.8 Million. Qwest is currently negotiating with several other prospective dark fiber customers for both routes. The new network expansion will be constructed primarily on highly secure railroad rights-of-way, thereby offering voice and data users a further degree of reliability over networks using other construction techniques. The Qwest Network expansion is planned to reach the following cities:
Mobile, AL
Bakersfield, CA
Burbank, CA
Fresno, CA
Modesto, CA
Stockton, CA”
[snip]
from http://news.centurylink.com/news/qwest-communications-expands-its-new-fiber-optic-network-in-the-southeast-and-california
Looks like Qwest layed down 12 hollow conduits (duct banks) at a time, 5 feet underground, along the railroad right-of-way. The railroad then leased the duct banks to individual companies who pulled fiberoptic cable through a particular conduit.
.
No idea how frequently a pulling station would need to recur to “pull” the cable through, however.
.
http://utahrails.net/sp/sprint.php
Is it just a coincidence that there are 12 conduits but only 11 cuts?
Hey pdaly – I spent hours digging around looking for past info about Qwest fiber installations in the Bay Area. I don’t know who bought their runs, suspect CenturyLink, but so much fiber was pulled over the last 20 years that I can’t believe somebody sought out former Qwest fiber unless they were looking for specific content they knew ran in the “heir’s” fiber.
The fiber runs that were cut also appeared in highway right-of-ways — I think only one might have been in or close to a rail ROW. The Fremont cuts in particular are interesting as they are all quite close, comparatively speaking.
Eleven successful attacks in the last year in the heart of Info-Land and not one arrest ! When I first read of this sabotage movement a few years ago I reacted much as you have, a minor irritation to the Beast, but your updated report seems to show they have sharpened their yellow pencils and started to make plans for a growing war.
Our Panopticon surveillance state may be effective in high density urban areas but outside those areas where most of the connecting infrastructure is located can never be watched effectively, this is why attacks on gas/oil distribution and electric power systems have been successful, again with no arrests. I doubt these people carry/use smart phones which the majority of the rubes dutifully use to assist the State in their own surveillance and suppression.
The Anonymous hacktivists who are in the same movement as the Monkeywrenchers suffered the same security failures but both have learned from their weaknesses and although security can never be perfect they have instituted rigorous security protocols.
These limited attacks may have one unintended effect and may force the IT infrastructure people to prepare for what a natural disaster such as the Big One may do to their systems.
“…but your updated report seems to show they have sharpened their yellow pencils and started to make plans for a growing war.”
At no time did I indicate I felt these cuts were attributed to “monkeywrenching” by anarchists. There are other players who fit the M.O. much better.
As for no arrests: why would there be one already? In an area loaded with technology, why would anybody on the street look twice at persons who might be disguised as network technicians going into a manhole? Especially in the dark for the first nine cuts?
I would also be careful about conflating Anonymous with anarchist “monkeywrenchers.” Very different motives and modes.
Livermore? Livermore! People, the US Sommelier Association has headquarters and a school in Livermore.
I think it’s the French. It must be the French. They have those little knives and everything.
The reactions and attempts to rationalize these insurgent attacks are almost as interesting as the actual attacks. The comments at news sites about these attacks range from spies to scrap thieves with a few references to ‘eco-crazies’ and an anti-monkeywrenching site proudly displays the fear inducing Eco-Terrorist brand.
First the Authorities tell everyone this is Vandals, don’t panic!, then they ask for junior G-Men, the public, to turn in their neighbors if they appear suspicious, this should produce interesting results when every service crew is interdicted when they stop to do their work.
Then we have the Detective/Crime novel meme with ‘perps’, ‘MOs’, and 8×10 glossies with circles and arrows to study for clues. Because this is in Cali there may even be a movie to spin out of this story.
The Spy/Intelligence/Espionage meme certainly has movie potential but the Chinese and anyone else who really wants to, have already shown they can access almost anything they want through the web without leaving the comfort of their home base.
For those who are not familiar with Light Pipe technology you don’t tap it with an axe or saw, any significant attenuation of the signal immediately sets off alarms. Tapping is accomplished with sophisticated optical, non-invasive equipment that requires time and expertise to install and monitor.