← All talks

Bypassing malware analysis sandboxes is easy, let's discuss how they are doing it and why it works

BSidesSF · 201738:536.7K viewsPublished 2017-03Watch on YouTube ↗
Speakers
Tags
CategoryTechnical
StyleTalk
Mentioned in this talk
About this talk
Bypassing malware analysis sandboxes is easy, let’s discuss how they are doing it and why it works Have you ever received a piece of malware and wanted to know what it did? You may have used an automated cloud malware analysis sandbox like VxStream/Reverse.It, Malwr or built your own Cuckoo sandbox. There are also high-end commercial solutions available such as ReversingLabs, Lastline and those integrated into Email, Web proxies and Next Gen Firewalls. What about malware that uses documents such as .DOC and .PDF files versus a regular .EXE binary? Some sandboxes do multiple file formats, some do not and some claim to mimic user behavior. In comparing commodity malware, what we get in email and drive-by surfing to the advanced custom malware in targeted attacks I found some interesting things while doing manual and sandbox analysis. This talk will look at free, commercial and gateway (Email/Web) solutions and what can be learned from comparing the results to your own manual malware analysis. Are these sandboxes worth it? Should you use them? What are the gaps? How much should you rely on the output? Do these solutions provide us what we need for Incident Response? Or enough data to improve the defense of our networks? Do they give us enough artifacts to remediate the infection?
Show transcript [en]

hello San Francisco all right how many Austinites here other than J Gore and Alec upstairs you got any of them keeping it weird all right come on to my Austin hat on I'm a foreign Californian so it's great to be back it's nice to finally speak at besides in San Francisco I've spoken a lot of b-sides but I've always wanted to come back to my home state and say hi to everybody and share some of the stuff so we're gonna have some fun talking about now our sandboxes are they as good as our analysis I am Michael Gough now archaeology is my personal website where you can get the Windows logging cheat sheets and good links to other malware

apt reports that I've collected that I think are good in regards artifacts how many people here know about the Windows logging cheat sheets hands okay got it second a couple half all right you got to get some more use of these things so go to that site look them up it's just a side note here I'm also the co-founder of lug and B I'll talk you actually see some of the information in the presentation that helped us find the flaws with the malware sandboxes and we were at blackhat Arsenal last year so that was a lot of fun so who am i I'm a I'm an austenite now but I am a Californian by heart I'm a blue team

defender ninja I am a malware archaeologist I said the archaeology in college well it's not really worth anything in a real job so now I put malware in front of it and hey it works and why we all here of course no one wants to be on Craigslist for sure I'm also a log a holic hello my name is Michael on Malaga holic really decided the room the only one that got that joke really Stratos again I loved properly configured logs they tell us who what where and when and hopefully how this is an area in our community that is really quite pathetic I spent three years in the gaming industry fighting the Chinese hackers if you want to read the kind of

stuff I had to deal with read report by Kaspersky and others on win nti and the one thing that actually saved us was the fact that we had and I've been working on this since I worked at HP for eight years with Fernando Montenegro fees around turning on logging and properly using it we had done this and it was amazing how much stuff we caught of the behavior from the Chinese attackers where all the other security products failed so to me they're incredibly important they should be the number one priority for you guys going forward to get these things collecting what you want you might see why here in a second I also created the

malware management framework think vulnerability management but with malware you read the apt reports you look for the interesting artifacts you put those in other tools you look for those things and then the cheat sheets and if you listen to breaking down security anybody LOI Brian and Brian aren't gonna like that so go take a listen to the podcast Brian Betz you're the co-host is my partner with the log indie project so malware valves right so we must evolve as well or as Neil Darwin says we must have much evolved if we don't evolve we're going to die well in the case of obviously InfoSec we're going to evolve or get breached right and getting breached means in our GE

anybody our Gd yep but first thing I tell people and instant response was the first thing you do Mike I update my resume its fact we are scapegoats and we often get fired because we didn't catch it in the act I have unfortunately found the largest data leak in Texas history they then fired my boss and made me boss so what did they just teach me if you find something you're fired so let's talk about sandbox is a bit so let's define what I think a sandbox is a VM you build to evaluate malware I know it's a premise virtual machine like cuckoo now and cuckoo sandbox anybody here play with cuckoo or any the other

sandbox solutions the same thing a specific malware analysis ecosystem like REM NOx is where you create a virtual network environment to analyze malware a cloud-based solution like payload security or reverse reverse tighty last line mark on etc all these are cloud analysis solutions email gateways that have plug-ins for this things like fireEye cisco amp etc wildfire such a web proxies like fire like fire I and last line and then advanced features and firewall so these are things that as the payloads come through they'll send them out like fire I does and detonate them like Palo Alto wildfire product and of course anything you specifically build so we're going to talk about this and show you some weaknesses in

some of them so let's talk about ways the bad guys I'm not wearing into my call it ways they bypass the sandbox technology first thing to do this is I'm going to talk about the technical things first there's a lot of options here but these are pretty well known they're people have done articles on and I don't spend a whole lot of time on these because I see the ones that I talk about a lot more but obviously VMs have vm tools and some people say well don't load your vm tools an hour we'll see that not run one of the things i do is i do training Bower discovery and basic analysis training a two-day course and i

give out a piece of malware that does not detonate on any vm parallels not ESXi not vmware not not VirtualBox none of them and i do that on purpose because some people in the class will run virtual appliances and other people will be on bare bones which is what i recommend people bring and the people that have bare bones the stuff runs that people have VMS do not and they're looking at things like vm tools the registered keys that are unique to these various VN solutions they look at those and say do these exist if they do then i know there's a vm and they'll stop running the hardware for sure they look at because you know vm is running on top

of the OS there's a shim in between there to do the translation and they can look for these indicators and determine whether or not they want to run we saw this with the chinese i detonate a malware lab turn around and turn around again one minute later and boom all the my works just gone and then we finally had a bare-bones lab we put it in there and work perfectly fine so when you're dealing with people that are advanced they're going to know all this stuff so yeah but you have to understand that bare bones might be a better option recently even been seeing an interesting case of ransomware we have a lab we detonate ransomware to see what kind of

IDs and what kind of interesting thing into doing these days and we found ransomware starting to look for the recent file activity so if you open and close a bunch of documents recent files is populated and updating roaming whatever it is and you can see all bunch of documents and things that you've recently opened in a lab they're not populated with anything so we actually saw this and remembered reading about it and we threw a bunch of files in that directory executed the ransomware again in Iran just solely on this item alone so an easy thing for them check out they're looking at the system to see if it's real or if it's virtual so they node either execute or not

excuses they want to be evaluated a big one here is processor related indicators some API calls take much longer to VM because it has to translate through that shim to the original hardware and the timing of that return really can't be adjusted by URI okay we can't really do anything to it to the system to speed that up so they look for those specific things that are making calls of the processors as it hops through the translator back and forth and they can see the timings off and they'll know the systems of um and they'll not detonate so one of the big ones we see quite often kind of a funny one is password-protected files so you have an

email gateway they put a password on it as you can see here the password is in the body of the document this is a really simple way to bypass all the checks because it's encrypted a me can't look at it the event malware solutions can't look at it because it's locked up now real simply if you just auto automate this stuff as you can just do a string output of these files if you want to batch process these things in your environment and you'll see that encrypted package occurs what we started doing because of this scenario is we have our email gateway for us a copy of these so we can look for something like

the sample of email and then we actually take it out detonate in the lab unfortunately we also have seen a behavior of users where even though you tell them to encrypt you know PII well they do and some of our people the sender's coming into our organization or also going out will put the password in the body so this is a somewhat normal behavior which means if it's a normal behavior by the user they're going to click on these things and actually put the password in and so that's kind of a drag so definitely tip number one monitor from your gateway or exchange however you want you to figure out 80 will give you a great indicator so you've got any

Evie usually will have an log message if you put it in slunk or whatnot that the file is encrypted make an alert or a report and look into these files to look for this gap because this is how they're getting by these sandbox analysis okay that's another one it's this is more recently I found this one rather interesting so initially we would see a document that opened up with a no lay object versus the macros because again macros once you turn them off in your environment then find another way to send you stuff and Olay is one of the methods but I saw initially was these documents came with one Olay object now we're starting to

see them with four the cool thing about this is these first two particular pieces of Olay objects in here are benign so what they're taken into account and we tested this payload security for example initially did not protect the Olay objects they've made a change to detect a first Olay object but when there's four do they know to go to the first one and the second one then the third one the fourth one in this case the last two are malware speedy scripts at launch the first two are benign so the sandbox is open the first one go through the loop and drop out and never even go look at the other ones so that's another way they're bypassing

it's something we can look at and again string it out you can see that there's an old layer in there and something you should pay attention to and make some daily reports to investigate and there they are another way I'm seeing the bypasses stuff pretty heavily is putting a URL in a document this is not new but it's a great way to get by the sandbox now some sand boxes can read this initial URL and open it but it will take an auto execution of that URL being opened for the sand boxes to actually analyze what happens and we're seeing that when you click on the URL it takes you to a web page and now within that

web page there is a place you have to click to download the document and then open it which is usually nuclear macro whatever and so the sand boxes don't know how to deal with this okay it's just it's a bummer so you have to watch out for these this little sever one how do you how do you look for a URL inside you know email gateway how do you look as URL we're pretty much going to use the spam in the outbreak filters to deal with this so this is one you should definitely take note of as well because I think this is a clever way if you have a web proxy obviously that helps so you

can do some stuff there but what I like the most I think this is awesome and I'm going to spend some time on talking about this and why it's so cool is time they know that in the course of an automated sandbox scenario that they're just going to wait you out if an automated cloud sandbox solution you upload your document and it sits there and waits for a period of time in this case 250 ping to Google DNS which might make a good alert right how many pings out they can use any IP but if you have a you know certain ping to an IP address more than if you staff count by greater than 20 and smoke or

something you might be able to detect this behavior but this time things simple for them implement because of the many ways to do it and what they're doing is they're waiting long enough and they probably tested it by uploading the file to these sand boxes right they use the same technology we do they break it one of the reasons you know not real fond of using the technical proof in the technology that you have works a lot of us don't they know what the flaws are and they're exploiting them just by waiting 250 this is about a 10 or 11 minute wait with 250 pings and so the sandbox automates it puts it to queue it

launches the malware it has you know some amount of time before it drops out there are some solutions that take 4 to 11 minutes to return you the results so 4 to 11 minutes exactly what this ping is trying to avoid so how long can you wait how long can the automated cloud solution wait imagine if they waited 30 minutes we've had malware that actually had a 30 31 32 minute wait before it did anything automated solutions on the cloud or even your fire eyes or your Cisco's your wildfires they would go dead you'd complain left and right if these things had to sit there for half an hour before they returned your results or they held your mail for 30

minutes while they waited it out so it's really not practical for the sandbox has automated solutions to wait so this is a big drawback now it's a very clever way for them to get around it so they'll wake you out they'll just wait for the cue to backup and then again how long can you wait and so here you can see this is a payload reverse audit or payload Security's free solution where they can see the fact that the network related traffic went and did ping went outbound runs a shell command exe calling out for ping and sending it to null so nothing's chosen screen and then they spawned ping right so they caught all that in the course of doing it but

they don't know that because they see this that maybe they should do something or act something differently so this is a challenge to anybody is doing cloud solutions and developing them out there hey guys they know how to get around your stuff just wait it out I think this is really clever and I've seen it always whether or not it's just a call to a URL looping or the sleep timer isn't everything but anything time-based is really a bummer for automated sandbox and again you can see here in the course of the of the cloud solution it saw that windward we open the document it launched W script with the oleh object pretty typical it then opened the

command window and pretty typical it then pinged out for 250 and then W script executed this document part dog VBS but that's all it found it did not find any DNS requests contact his hosts or HTTP traffic so again not only did do online cloud solution give up but so did our email advanced malware protection but we did not give up and login D caught it cut it all so you can see here the same information that clouded provided us cloud providers you know basically the doc opened the what W script opened the ping and then ran the ping 250 times whoa hey what happened there that you guys are me looks like wiggling have to hack the hardware come

on now

houston we have a problem are you picking us with you come on I'm from California be nice here you go did you wiggle something he wiggled I'll fail wiggle it again there it is just stay there Oh No went away definitely you know nope hold on yeah wait for it there it is don't move what the heck really nobody else had a problem like this have the cloud guys of the mellow Aryans broke in here and mess with our video with the heck is hilarious

[Laughter]

you

yep

doo-doo-doo-doo doo-doo-doo-doo dude are you I'm plugging in a blink so those are something there you see mine yay no all right Chucky's it worked for that long announce incited to stop what the hell

join em plug it

all right

it's it's let me know if it goes off alright so the cloud solution found these items and we keep looking behind my back now nervous tic announcer Texans so the cloud solution found this but log MD found after the 250 second wait time doing manual analysis however you want to do manual analysis but we use log indeed and it caught the fact that 250 of pings later that it used run DLL and loaded a deal of DLL on startup and with that we also found the network traffic that they did not in the cloud solution so really really

doo-doo-doo-doo-doo-doo-doo any questions tell me was this this point I think we're calling for a beer break everybody drink

you

the question was what percentage of icy Mauer checking the seafront on a VM I'm reading numbers as high as 30% I would say from the samples I have that that is pretty accurate for more I think it's becoming commonplace I think they're starting to include that check in the libraries depending now granted if I wanted to infect a me in like you know Amazon I would write my stuff to look for that its Amazon and I would inspect the box specifically for that VM all right cool for now rap and running I guess you'll let us know if it goes down so one of the things we caught in the course of of the cloud solution did not

show us any network traffic when we do the malware analysis after waiting it out we do see all this traffic from all over the world anybody care to guess what this is tor so this particular malware is using tor to phone home so all these countries are involved now again cloud didn't see this but if you waited it out did manual analysis after 200 pings give or take or even the 32 minutes boom you'd find out that you now have lots of traffic and again the persistence is another big one right you're looking at it like a snake the body is some of the stuff we find the artifacts the files the payloads directories but we also need the head

how it does this malware persist it's real important to cut the head off a snake when you're dealing with Mauer and Incident Response and so here we can see we actually caught the run key and the DLL loading up like we saw in the process executions so another thing that our manual analysis process catches let's talk about a disposal provider that had a serious flaw I first announces that derbycon and I opened this up as a first as a service help ticket and it turned out it there was a bug and there's I have three open bugs with this particular product let's talk about some of the some of the flaws that occur with these sandbox

solutions so hey got a fax seriously so 90s still get packs of today it's ridiculous it was attached Word docs so you can see from the date to give me a perspective of time of when all these kind of sequences happened they all look like this they actually was a screenshot of an e fax now I'm in real basic Mauer analysis we can open up if you don't know how to do this so if you haven't done this before Word docs are just archive files if you open them up you can see the fact that macros are in there if you click on that folder you can see that it contains a VBA at this point I know something Bad's in

the document pretty straightforward I can do this at the command line as well with the 7-zip command line stuff you can see it contains a macro so immediately the cloud solutions should see this before they ever try to analyze it and say hmm I might want to think twice about this and of course at the command line we can look at certain strings always you know again if you just string out the docs you can see the word macro or macros in this case and macro sort to select whether the law is in the clear text string of the word doc and then also the infamous doc open that's what's going to happen when it

when it opens in the contents enabled it will auto execute and again just using strings I can find this out pretty straightforward and then force office mail scanner where you check out Word docs and you go ahead and look at it and even office mal scanner saw it was malicious in nature and you can also extract the BB script using the info tag in office Mel scanner so there are three things I did take me less than a minute and then here's the fancy-pants email gateway solution looking at the file we can see outbreak filters none so there's this is not an outbreak we can see that the anti-spam engine sees it finds negatives we can see the AV does not yet

know about it and so it's clean and advanced malware final verdict clean so I'm saying it's really

it's like when it relaxes it dies

what well what's my failure sir all right so clearly the tantric dance solution says the file is clean like me I don't think so and a couple more that came in later yep clean yep clean so at this point I know I get a problem with the solution these were bought tract variants even after the end of the day same-day McAfee knew this thing was bad because we started having users click on it and boom McAfee would trigger and again still says it's clean in the same time that Mac and he says it's bad on the endpoint and that is about track again in one minute less I was able to tell you this word box malicious a nature

using 7-zip strings in office mail scanner to be certain files bad you should always detonate it in some sort of lab solution but again the Saints fans cloud solutions have timing problems so it's not really good way to do it and when you start automating all your documents there's a challenge here in that you might be sending tii with valid documents so there's a warning for you if you're in that area where you can't do it so even virustotal after eight days knew it was bad pretty obvious and the same word doc with Olay object comes back is unknown so it went to the Fancy Pants solution because it's Olay objects it doesn't know right doesn't come back as

clean doesn't come back malicious it comes back and unknown so we look for this to take off as well in our in our log events so again bad so this is I open a ticket for this vendor and again for the Olay problem and they're working on that one as well this is coach or if you're not from a filter it's some nasty root kidding it's got no bytes hiding their stuff in the registry the payloads are hidden registry so telling me it's unknown is unacceptable so let's see what cloud analysis says so here's a verse tightiy it drops the file oh i eight or nine dot exe yep he's that unusual characteristics yep document open just like I do in strings

so we know something bad there's going on we can see it extracted file yep it showed us that earlier and we're clips and definitely launched some weird strings open virtual Alec XOR all triggers to me to say something fishy I'll speed up a couple these slides to catch up on those blank screens so reverse side it definitely shows us when we're open oh i 809 and it spawned a second copy of it and again no D or contacted host network traffic from the cloud payload so what do we want to get out of analysis what's our real goal here URLs right via web proxies we can look to the URLs and web proxy see if

anybody visited websites the IPS we put them in the network device you know look at your network logs firewalls but not in Splunk and you want to see if anybody else visited those IPs file names and directories you know autoruns config changes metadata signed or not signed who's the digital signature or lack of digital signature D behavior what was the process tree looking like and of course the net flow of traffic behavior is really important as well these are the things we want as IR people are as detection response people or hunters artifact hunters so why do you month is data well obviously you wanna know who else got infected who else open these documents be nice we can block them all

but the reality is there are flaws in these products so therefore we have to keep up with this and make sure we check to see if anybody else copies what was added to the system what was changed so we can either remix box nobody becomes reimage the box or clean it up where are people we have to clean up the stuff if someone says you got 100 machines I can't reimage a hundred machines it'll take us two weeks and all this people won't be working so you got to clean it up all depends on your there's a there's a climbing here this is really funny right signing of the string going out just like diving attack on the video

hahaha wait 30 minutes it'll come back here who said that you get a thumb drive he says hey you had a thumb drive really good that's a question earlier that was way off we try it again that was close it says warning this ad may can tell malware man right on it so let's look at malian manual analysis what do i do like a little script that tells me hey went to shop tacky so no brainer windows isn't very good about doing dns but IP config display dns does it also tells me that it's Russian Federation I know that's not good and also it happened to be using Google Drive as the place that retrieve the

file I reported this to Google to take offline because was copter and so that's a bad thing but I can get this information manually the clouds kept the cloud solutions automated solutions can not and so the parents IDs will tell you if you walk down the tree that they launched oh i 809 we saw the same thing the cloud did here with the difference of Alexei 50 that sandbox on this Alexei 50 spawned went seeded system 32 windows 32 dot e XE we didn't see that in the cloud and it deleted the original payload and then went ahead and spawned a second one just again log and E or manual analysis will find this but not

automated cloud solutions and again tells us the pig so we now know that these things are Google and Amazon we know who is talking to we have a minus W option where we can do the who is of all the IPS which immediately gives us all the next country data and network owners and the range so we can block if we want and again the cloud analysis did not see when host 32 so there it is oh I 809 and look down here the windows 32 you can see is acting as again the behavior it is acting as a browser they but trust me it's acting like browser wow wait for it any other questions

yep it can but you have the option you're in control of your own personal built sandbox where the automated solutions just drop out you're not going to ever get that thing to give you the info you want so the question is is a strategy or how long should you wait generally if we don't see the conclusion that we normally see an hour we will go ahead and launch it before lunch go to lunch and come back and redo it and generally we'll see the data we need but an hour I haven't seen well actually that's not true the Chinese apt for the win NCI project had a backdoor that waited 31 days okay so that yeah bummer

no I'm going to unplug and plug back in just to see yeah it's definitely not me all right question here

no a lot of malware question is should you make your systems look like vm's it's kind of the same thing load pcap everywhere and maybe the malware won't run because it thinks it's being snipped that's obscurity and it's only going to work 30 40 percent of the time a lot of malware doesn't do that and you don't know what exactly they're looking for and in some cases like the timing doesn't care the timing is the best way just to break any automated analysis so good question all right we're up against we can get through this thing all right so yeah go back here so you can see that this thing is actually being talking to the browser

something behavior that the sandbox did not see just hold it hope over the laptop of you really got another laptop about a bigger table I think it's a size issue well it's like as soon as you touch it it jump all right so we can obviously see now that the run key is launching Windows 32 just like we saw on the last one so login D definitely catches that and you can see here that you can see that oh I 8:09 spawned Windows 32 and that they have the same hash so it isn't exactly the same file so they no cloud we'll see oh I oh I 809 you'll go look for it Oh a oh nine and it actually what you're

looking for is when I was 32 and and it's not going to be there you're gonna if you're going to miss it this is this is a perfect example of a tactic they used to avoid getting caught right there so you can see the hashes of the same you had to look quickly how we didn't miss it I think that box partied too much last night thinking give give it some more beer maybe it has a hangover I was just trying to be funny any other questions while we wait up video cool so another script that I run I look for parentless processes here's a tip again if you're doing looking on a box there

are some natural Windows processes that are not don't do not have the parent currently loaded because of the boot process but after the system boots parentless processes are very much something you should look into so obviously oh I 809 deleted you saw the null deletes and all and now this is guys hanging all by itself as a parentless process so I just look for those and I upload those as part of a script to virustotal to see what it knows these features will be put into log in D because this is a real valuable item but definitely look for parentless processes use process Explorer or anything so let's compare manual to cloud what we saw this far so URLs we

got some but I'll give it a yes to a cloud and a No so we got both IDs they didn't show me I gave you two examples did not show any information at all some file names and directories again spawned another one they never saw the second one and didn't see the DLL after the timing occurred and autoruns nope didn't you mean the conviction jizz nope gave me metadata told me it was unsigned behavior unsigned document and it did not give me any behavior like I would expect and we get all of this with manual analysis however you want to do it so sandbox Emmanuel paid solutions do work a little better you get a little

more information but it still would not fix this particular these bypasses many samples definitely fail when you try the XM NB M so again I mentioned the fact that I gave it people needs out and training to prove my point and again not as much detail as you can get on your own so spend some time build a box add it to the domain and take it off and put it on an isolated network we're finding now M hours looking for the systems to be on domain or they won't execute as well so we have an on domain machine a domain machine and we do them both we don't get the proper execution we go

ahead and do it and some are even looking for network connections so you might have to to and then create a network connection map a drive letter and again sandboxes are good for multiple samples if I do a baseline then get one good answer on one payload sample I can then run for more of the same thing and I'll know they're all the same thing if they look like my results so ransomware this is really handy you look at the ransomware you see what it did upload the rest don't bother doing it also the amount of time it takes me to do manual analysis with log in deal of the reports sometimes the sandbox has

still not returned me the results takes 20-30 minutes sometimes with the backlog and you may have some super hard near VM to get by some of these tricks that they use but I don't think and get around the process stuff the process or process or stuff I'm just not going to happen so what do we use we use login dates free there's a freemium toodles like that's like burp suite Sephora Mia model the out of the system so turn on the stuff that we need you to collect we have the ability to harvest log data but also do things that shot 256 deep we can hash the full file system or read shot we can

take a snapshot of the registry we look for large reg keys and things like that and so you can use this to build your malware lab and test your solutions and see if they're actually working and we have a pro version as well like birth Suites 200 bucks per admin is pretty cheap we do some additional reports and harvesting system on logs so that's how we get hashes when you saw the hashes in there we have a master digest something about his unique files vs. hashed baseline which is every file on the disk it's about 40 percent smaller if you if you do that and that they're sorted for easier lookups and who is stuff that you

saw me see that that was Google and then with Windows eight one and ten there's a shrum database system resource utilization monitor can tell you the application and how many bytes in and out the application made so once you figure out what the binary is I can go look up Windows 32 and see how much data went out and so I can now measure not only the potential loss of data but then I can go back in time within 60 days that shrim gives you and I can tell you in the Box was first compromised and you get free updates for a year and and emanuelle so again PowerShell details we have a report for that power shells

being used more and more in malware Metasploit stuff as well use pretty heavy in PowerShell we're about to release an auto runs report in April and virustotal stuff lookups as well so we do some really cool stuff and add the parentless process and all the white listing that we talked about earlier so what do we get with all this manual analysis gives you what where who and the details to improve your active defense and each one of these samples I gave you I do it in about 15 minutes faster than the sandboxes returned me obviously I'm efficient at it's what I do every day but you can get pretty efficient what you stick me an hour

takes me 50 minutes well you just taking 2-3 hours takes me an hour so it's a big time-saving doing this and I don't have to wait for the cloud solution return stuff and again at the level as your level of experience you know starts coming down you're going to trust these solutions more use manual analysis you'll get better results for sure and that's it with all the interruptions I'll take questions on off stage and those who ask good questions will get a thumb drive all right any other questions go back and do this and that's your action will take away here you can actually go back to work and do this not theory

you

so the question is what's a typical example of a bedlam systems whatever your last PC run is in your corporation just replace the hard drive with an SSD small SSD and we duplicated we have a disk duplicator Alera Tex what we use and we just have a bunch of duplications with SSDs so we can quickly take 15 minutes to dupe the disk and we can rip through these things like nobody's business it really works well we have a master image we update it as we need to put it in a drawer and then we start duplicating it it works really well but you're your last version of your own hardware you don't need any power to do

manual analysis there's just no process intensive stuff so good question ok so talk to me outlines we got to forget it so thanks everybody we can see those yeah looks pretty easy to it so once again follow up with Michael I'm Bob peerless give any feedback for the conference itself on sched which is the application in phones for doing the scheduling of seeing schedules on behalf of b-sides and fitment we'd like to present you with a Fitbit Alta and remind you that motivation is your best accessory also we have a I would also like to thank our major sponsors hacker 1 and Fitbit thank you very much the next talk should start in about one

minute [Music] in from another land across the novel Daltrey's know [Music]