Digital Energy BPT (Basic Persistent Threat)

identity twist down smart breed infrastructure Network infrastructure can put in the support smart grid mode digital energy and originally I named this talk BPT for basic persistent threat out of my severe frustration with what was going on in the community that everybody getting hacked with faces specifically in the Telco service provider utility skating environments then whenever anything happened

I started presenting this Carolina con back in the spring equipment there's a Common Thread it's all about the vendors the vendors are the issue so it's a basketball machine out of the field the turbo infrastructure is is if you can get one basic really using hand hold into the network the trust relationships are so wide open that you can pivot add wheels and move horizontal work into the network with no challenges once you get that initial handmade Defenders have got wide open remote access that you're not going to to have any issues getting into the network component because they don't trust their customers to be able to run the medical systems that they put in uh they

basically uh you know support the local technicians to keep everything underwater so you can pop it in there you know accessing the network really easy it's not an APG on when pla for all this stuff even if they may be involved sometimes and the big issue is the flatter philosophy you know big flat non-segmented Networks and uh so here's the kind of networks that I spend my time in I mean their architected in a what we call a Next Generation Network for uh confirmed service provider Telecom where it's getting in the marketplace is getting very brave between the Telco the utility the cable company the service box because they're all into the quad play The Voice video internet wireless

tomorrow's back home providing all these Services open in single infrastructure the only difference you have many utilities where they're in the water gas and the Power Systems as far as that they're all running all these Services over at the fireworks that they're going now like the utilities are running out fiber to the media immune cases and the Telco was maybe even providing the smart grid infrastructure to the utility and but what you have is at the very top is you got what we call it in the business in the Telco world uh the OSS MBA says the Operational Support system and the business support system business support system is more like the Erp or the Telco

service providers where you have all for the real business gets done and the Operational Support Systems we always says have been through the top those systems is how things get provisions and moderns have been managed and and then you work your way down into the clouds you're getting the core infrastructure where you have your core routers you might have your Optical dwdm infrastructure maybe if you've got a new upgraded networking maybe you know build them Optimum or if you dynamically manage and dynamic allocate lambdas across your fiber and provision and Optical wavelengths on demand and then you work your way on down and depending on the organization that may go with a passive entrepreneur for an active DVD Network

or it could be still an adventure of the Legacy team TDM sauna stuff you'll see a mix of these things but what's important with this that over on the right is every as you work down through those layers and all those different systems they all have a union vendor all of those vendors won't remote access into these networks to help their customers keep these systems running and they open up a very broad attack structure so when you go in with renting to go in and do an audit whether it's PCI or or one of the other compliance faster they're just getting a security assessment for insurance purposes uh what we're seeing now is everybody's concerned about the

records coming out of the White House they're afraid if it's going to have a mandate not voluntary suggestions and try to do security so they're getting uh get breaking and renting to test things when we're finding is there's a really broad attack Services coming from the internet in and once you can get a basic handful of any of these Services you can pivot and it will into these uh assets because of the lack of segmentation and lack of a mind of security and I get to go into some other people some real world details here examiners the same vendors used in the United States as as are used in other countries yes I was just in Australia last week and

that deep Intel having the same conversation foreign we're all going through the same schools and getting the same training you go again Cisco with your training Microsoft training we're using the same stuff and nobody's teaching you in any of these courses in the trust relationship the interdependencies blockages of horizontal movement they just teach you how to you know get Bloons of Oxford so with the state of this is an example this is a diagram of the ANSI Isa 99 standard or stay there there's International standards body for industrial control systems and scada and that diagram on the right is how they suggest things get implemented with copper segmentation between all the different layers of the architectures of

firewalls in trust relationships with lockdown need to know that's a real knowledge diagram that is far from reality yeah somebody's probably doing this I haven't ran into him yet if this was implemented it was properly configured with proper egress which is another foreign concept the egress because but in reality what we see on the ground is where you see Fireballs in reality movies see if you go into a uh if you're part of a red team you go in and do a compliance has see how far deep in the network you can uh can can get in what you're going to find is in many cases where they have firewalls here they'll have like a literally an nt4.0 box if

you're if they're lucky they got an XP box I'm serious you can't make yourself up hpe multi-comb that's secure if you don't believe in Nashville if they don't have a real real red face and argue either that this is secure because fish together and it's a secure don't pay a lot of money for it another example with so firewall stuff here in the and while I started coming up with the vendor persistence threat console with scanner one of the examples I've seen with a utility went in and look look at their Network the firewall was configured Cisco firewall permitted any of those in and out from that in fact you know I'm over here you know about to

pass out you know getting a lot ahead firewalls that's all in the Box I had to go out in his way to do commit any out and this was offended they put this in a building system they put this you know Cloud I put a name Cloud experts through all the main services are in the cloud and they put a firewall then for to protect their local land so after I uh I recovered from seeing this firewall coming through with a with my voice cracking guys manager please show me your state assistance because this was on the business system and I'm not sure surely whoever did the state assistance did arrived it gets better so I'm asking

Andrew to show me show me your studies and how do you access faster than my eyes can keep up with equips at his Android phones no password hits B and C and has a scada in my face my jaws probably winning

by Civic and patriotic Duty dude I'm here to help you I'm not going to take advantage I'm really curious and he gets married we think this is and I start trying to show me one more time slow motion pull out pull the phone out of your pocket and he's been very very angry because another thing here put this but this is that the Memphis Cisco we can inbox we'll set a conversations if it is responsible because well our guys love it because when they go here honey they said the tricks they have to make the monsters I can sympathize that I'm a rabbit Deer Hunter I'm going to be therapy because everyone missed Uncle David you're

seeing in Tennessee he's full of durbanon so I appreciate that but it's like yeah so that's what's that is what's happening and that's what's not hitting the news and nobody wants to talk about it two cases one bedroom put in a firewall permitting another one put in the firewall in front of the skating system with the doe passwords and vegan single Rider um and then another example uh where after we went in and did an audit to a particular Network many years ago and gave them an architecture review and told him how to think they never could put the fireballs and stuff in place and we came in here about four years later and did another otter there's so

many hoes in the firewall that all you have to do is go in and hit them with a fishing attack and the fireball's got so many hoses you can freely cortisol and correctly pick it all through the network now just I'm glad you can get the ballrooms and it's just the best of your router I mean I mean it's just so wide open so that they can work on their desks on the Enterprise Network and access all the critical infrastructure but but if the monitor comes and looks at it well they got a car so that's important really going in and uh as a country we really need to go enforce and mandate prevention somebody needs to go and do this to

these networks and make sure that we're not just checking boxes and looking at Blinky locks in the rack and looking for brand making ideas against Cisco's jewelry firewall so this here's an example of a put a soft switch number especially with sauce which is you know if we got a Cisco call manager in your Enterprise site concept or a Telco but it's a system that we Consolidated room cooling equipment same size of this room there's roads and roads over the class five Telco systems you can now consolidate down to your a less than a half a bag of board systems to run a large Regional geographic area and we call it a soft switch or the Telco Road

and what happens uh with the sausage pepper in many cases they know that they're going into an environment that has not changed since the first system you got put in 23rd years ago and they go and drop in all those new IP technology they got to help again keep the guys running and help them Division and help them troubleshooting the motor and so what they will do is uh drop in a a multi-home Linux box or something one leg to the internet and one leg into The Voice network continue to trust you or its core infrastructure as in 9-1-1 dial tone or Emergency Center and what we've seen is is those systems you can get put in

and told the bypass firewall in some cases without politics there's not a corporate security policy I I hate compliance paperwork ought to do

and then we get called after they get notified by the bank of England so 60 he says this IP address is a type in our bank and turns around our system as their core voicemail and they're lucky in strip creating not an advanced adversary who would just don't pop the box and go sell them and fix it lock it down and just wait until they to a more appropriate time to do something with it here's another example with the curriculum structure we start going in and doing auditing and testing and put me into the network get into the provisioning systems and the network management the elephant management system what we call the EMS that manages the core infrastructure of

the network where we'll end up bonding is in some cases what's what what has been sold or what's being pushed out to Enterprises so it might be a tell code to sell the services to the smart reader there's what might be marketed as is a green topology when I was in the Enterprise world I thought a ring was the ring a geographic reading what actually gets put in place in some cases that you got to be very careful about if you're on altitude and testing and look at technology is uh is it a when they say they have a ring is it a collapse ring where it's a single bundle of firewood that runs down

the interstate it is electronically logically looped with redundant Hardware on both ends so the one backhoe one drum driver one scroll or raccoon uh or a couple of methods moving power copper can cut into the line and they take out take out a major Geographic period of lots of services and the reason I bring that at the method technology is I actually had a customer that had this this top architecture where they had a ring to meet SEC requirements for 911 you have to have an electronic logical ring well they had some entrepreneurial uh above us in their area decided that they were going to go into the copper business and uh time to climb their poles and uh

and thought that they were going to cut the wire and get caught but what they cut was fired and knocked down the connection between two major Geographic areas and that was the reason so that's something that when we go out and do red teams and Audits and looking at Network there's something to be aware of especially if that's the only thing they've got that's it is what it is but at least it needs to be documented and listen I just went under the uh you know that everything's good in the tornado or something comes through and knocks out making tough Communications so this example is this is another uh another building system that run up in the system

they're I'm coming to the point here recently they are the the biggest problem in critical infrastructure in the EPT in this case we had a Telco went in and uh did a red team event against their number Independence and uh started popping boxes popped their web server got a share on an Apache surgeon and what we found is The Shield that we got was older internal Building Systems we thought we were on their external villages their external abilities it turns out it was they both had their external billing systems and their internal building system they ran the whole financial operation all on one system and insecure if you don't believe it has been who is now on the security system

and so basically what happened we did the directory Professor to them got a shell

and we're ready to Pivot around into the network and by the way with that connection and the trust relationships this is what I've prepared down on the police is the express relationships that building system has a trust relationship to all to that box up in the right hand corner where the real business gets done with automated provisioning so that when you go online and you won't you won't new cable TV services you want to do DSL service new bandwidth uh you paid your bill or you didn't pay your bill the service can be automatically turned on or turned off without having to do a truck roll building system you can go with and gain influence and access

because of the lack of segmentation and security if you could go and get into the middleware for the cable TV and then to save the network Management systems for the transport and the routers and the switch Etc which we absolutely had it down into it was never uh services and in this case this particular Network they're also providing transport networks for sponsored so not only not only if you uh if you're an adversary you get to pop this in there not only do you get the get Telco for your hard work you also can get a couple utilities

really need to go bear down on these uh trust relationship between all these systems and uh what ended up happening here is the vendor not only put in that building system with that bad architecture with no DMC segmentation uh we tried to go into layering architecture and how that things should be done and we no longer do pay attention that's they become a pin testing company one of the things I learned is whoever runs the building system whoever that bigger is they went all working with school that's all of us in all of us techies that are involved with renting we've really got to get a business conversation before we can win these companies uh these matches with the

vendors that are doing this to the customers well you know that was Secure as hell people

because they really don't care about security don't tell me that I don't want to know the lowest price vendor whether they can really test together or Bernard is going to get the job and I prefer the one that's not going to break into anything like they're going to build system does not going to break into heat so here's an example um you know we always talk about the Insiders the rather there's no Insider threat okay whatever is an Insider uh incident in this case it was not intentional when it was accidental but it usually it usually could have been worse if it was in the kitchen Cox Communication a couple years ago had an example of this where they

had a router back but in this case in this case we have how the Telco and they they wrote out a big flat Layer Two core infrastructure out there in the middle of the network I think core fiber transport network is layered to one big fat layer two together I'm a router guy drives me nuts breaking things up the segment but that's what they wanted and what what this particular vendor does with this layer two architecture they offer they have they have multiple vlans have a villain located for boys for uh cable TV media uh internet and other services and any service that gets put into the VLAN that's dedicated for video they have dope the rscs for multicast it

does not go and do their whole ignp query reporting process it later too it automatically agencies it can be Institute into that feed land it automatically replicates it out across the whole network so you've got this big Geographic layer two flat Network and anything just put in that video network is just getting hammered out in the agency Hardware so uh so the Toco technician accidentally Provisions the internet being on over into this cable TV network and there's Internet subscribers are they're subscribers as well with heavily infected with malware and this was against Microsoft Rose when they put that in when they got opportunity

everything lays on top of that Trend that let that bottom make sure we can transport thing they call me uh with their heroin fire uh I'm on the interstate and luckily they have instrumentations in their room so we can come in out of hand and do look at flow net flow and Peak out that I had installed for them months earlier for a security solution and uh there's nothing left in on the interstate I hadn't pulled over and troubleshoot a pcap over your cell phone with a whole Telco panel don't recommend it it would take you crazy is trying to read that pcap and then come to the point where you realize that they have icmp denied a service

driver encapsulated in ignp their team travel I I swear I thought I was going to do this talk but I wish I had a Decap to show you because you can professionally learn ignp Layer Two with ICP encapsulated golf track and uh it's a crazy pink construction yeah we're following people it's terrible could have been a road uh employee or a or someone who broke in broken through and depending it over to the provisioning whatever here's a problem uh from a package infrastructure application that I'm seeing in a lot we saw this originally I discovered this with a couple telcos where somebody from an interesting activist arrest from somewhere else started targeting DSL routers one Telco they had before they stopped

in about 600 drivers coffee and they got their stuff because they had met though I'm a huge advocate pretty much data as you can out of band so that you can troubleshooting trust you these networks uh so that one tells I was able to capture it in the second Wing before they got his stop they had six thousand years old I was wrong and all the only thing that had to happen there is a is a nine to shift regional service you know control plane locking down the protocols on the protocols are going to communicate and then the management plan how the devices were going to be managed uh and then the whole data point they did not put the

picture into that into the separation of services which nobody does I'm still looking for a network that we're management and control plane data planes are separated and locked down hard like they should do

and then sure in here recently this same problem Brazil had they had four and a half million dollars on them before they got to stop and we'll see and with us here recently we saw it again just a few weeks ago where these DSL Ballers are being popped at this but this time it's more of a denial of service the first time the first couple times we saw it they were popping the routers and reconfiguring the enf settings that were being sent out in bmcp on the customer side and changing bring directly into DNS to ambassadors and another part of the world a lot more exploiter did the company have the management interface on the

routers up on the internet they had an engineering banks open to the internet and there was a basketball password I'm telling the defender persistent threat undocumented accountants I thought they had a user account password they thought they were good and then one of them 600 routers another six thousand Brazil four and a half million and then we just saw this here recently were some routers are being popped and they're doing something we haven't done the full forensics of it uh but they've done something to the firmware we're the routers are doing a denial service now they just kill them okay but when we go out and do Pit testing when you go out and go and go

and look at a time really paying attention to the Target that whole control plane management plan separation research that's blowing for videos for me I know where protocols they're just easy it's easy it's like it's like Microsoft you know communicable it's their AG some of those setups where the ISP still owns the motor limited the customer take it down a little bit high speeds ISP is on the boat s they may charge the customer but they manage it so here's the scenario again if you're out going and doing a head test against some of these uh crypto infrastructure guys to roll them out and converge networks offering multiple Services over their uh the new member of

the development per se for smart grid et cetera take a look at it was going after all these back all the systems because in this case I was at a utility and the vendors we're dropping in routers and Mel enough uh point-to-point vpns if you buy their systems if you're buying a 24 VPN some of them they might do a Remote Access VPN SSH but in this case they do they're secure

and through that trust relationship someone in the next area pit was able to Pivot over from the vendor network over into the to the back office Network where the assets of the infrastructure was and popped their box and we know that happened because the vendor showed up unannounced unrequested and started ripping their stuff out with no reason not even inside so it's

the question I have is how much motivity happened before they ever showed up and stop the attacker and whatever happened to that customer so one thing you'll understand the critical structure some questions they just don't want to be interesting don't ask myself what what I'm uh what I'm thinking about from a red TEAM stands one because I've only knew in the future of Engagement I'm going and show started showing people and what if what if uh what if there's an adversary was to go with targeted specific business you know press releases LinkedIn people crying about the stuff that they're working on you're going to say cop vendors vendor 2 there and you're able to go and break

into End customer Networks and then turn around and pivot back out and attack other information in their customers spread here from all those different customers and all those are different informs something to think about is renting to set something up like that with the customers will show them how this type of topic happens

in this case uh the uh the Telco put in a new transport Network cleaning this was put in place by a different side of the business there was two sides of the business there was the internet side who had people about security who made a huge investment in this security thousand and then there is the the boys transport Network side that didn't didn't care about security they just provide dial tone and T1 and T3

Network they required and demanded their own Circle be put in and not have to go deal with those little Security Guys and so what they put in place here is a multi-home Mondays box and uh one leg into the core transport Network that everything rides across the dimensions Department when they bypass the firewall and they they literally just moved they literally got a call from a bank in England that's how they discovered that an SSH approved Force have been conducted against that that EMS that were called held a Management Systems was conducted against that and uh the brim Force attacked from somewhere in Europe was successful and they turned around and pivot off of the

attack and everything okay and that's how they found out that that happened so if you go out and do your pen tested especially privilege so if you're going to you're going to see opportunities like this the old the old Relentless approved poor stuff it works

you know

eventually we'll get here thanks guys so so from a from a renting and former blue team standpoint which is a no matter where we go no matter what we no matter what network we plug into the layer two years you use for closer drivers being absolutely crazy if you can go and get any kind of access into that with a procedure to single literacy protocol Industries required if it was art created like the early 80s if you need to do that I mean for the left 250 Years and we're still out you can takes more time in the future than to own the network than art building and it doesn't matter where you plug in

it works and then the first hot redundancy protocols like the The Cisco guys with a hsrp or vrrp if you're not a Cisco guy or in the global load balance of protocol gldp any of those the IPv6 stuff all of that layer through stuff is uh typically a lot of them and neglected there's still low paying the food in these numbers one of the things that I'm trying to get customers to understand and learn if you guys heard launch meat too with the Xbox uh I'm trying to get especially with Enterprises we use option 82 in the Telco world for lawful intercepts from the bar tax and so basically what happens you turn on DHCP snooping on your Edge switches

so whenever a DHCP request is set the switch sees that and it will inject into the header of that BBC Cube request the name of the chassis and the invisible interface that that request is coming from instead of that to the dncp server so you know exactly where that IP addressed or press came from and who's guided and when they got it and when they go away you know and you get that by way of DHCP option 82. the problem is is if you're a Microsoft shop that lets micros unless the latest version of Microsoft Sergeant edcp application uh inmate support nothing I'm aware of sports opportunity or uh there's some Linux open source gmcp application it'll support this

optionable too so you can foreign

Port security all these networks have locked and locked down to these layers of interfaces so it's coming up there's no reason for it I mean everybody's involved and spent all of this money in our own new system and Juniper Networks and they're not using these basic knobs to turn to lock down into Layer Two um you'll put my defense stand for your recommendations in customers everybody's talking about applications what I want to do for these with these scada numbers these Telco members it's going to be in the control plane and the management plane and I love the networks and the Operational Support provisioning there's no reason that the traffic flow cannot be laid out and

defined and then blacklisted in the network infrastructure these devices are going to call this is how they're going to call this is when they're going to call it this is what the traffic clubs should be and lock that down anything else is not permitted and if anybody tries to the law of that instead of being wide open like it was today instead of just going trying to secure the hose let's secure the network and walk down the traffic reports and then whatever we lock down as needed traffic flows in the greater unit then go and monitor that and a lot of data puts uh what's abnormal they cut down on all the numbers they were having to

digest

questions

one of your comments sometimes a lot of times

for those environments in terms of being able to I know we have a recommendation coming down the do you think it's actually going to take you know