Disaster Recovery (Updated)

It had so much potential

Updated 12th March 2016. Newly added notes at the end of the post.

Back in 2014 I needed to choose a robust backup / DR solution that would help me prevent loss-of-hair & brown trouser moments - e.g. ransomware or user stupidity (my own mostly). All sorted and was fortunate enough to choose one that still covers my needs.

However since then I flirted with a couple of alternatives and ended up settling on a selection of cloudy storage options.

For the personal stuff I tend to use the big names (Google, Microsoft & Apple) which are linked into device accounts. These are really low-risk, low-value data items which business adversaries or other intruding agents [hackers] would find worthless.

However there's a lot of information which is business-focused - or that which is protected under one of my businesses ICO Data Controller registrations - which need more attention.

The reason I'm writing about it now is because since evaluating options I've moved away from Windows, so the requirements are now vastly different. So because I'm primarily focused on multi-platform solutions a lot of the offerings get defenestrated immediately.

Over the last few months I've been prodding and cajoling Tresorit to fix problems with their Linux client and have now officially given up. No responses from their support desk about quite critical issues in some time. One of the issues is that - whilst I had Visual Studio Code running, and doing some project work on a Python module - Tresorit started to sync one of the tresors which houses project work.

I watched in horror as the file list started reducing in number in VSC - it was like an unstoppable terror of code deletion. Anyone who's just discovered that code they've spent days or weeks applying themselves to is lost forever will know that feeling.

At first I hadn't made the connection between Tresorit and the file emigration but then I did a sudo find / -name <scriptname> only to see it right there. In a .tresorit/Trash/.. folder!!

Nope.

Not having that. Recovered all files - and a bunch of others I hadn't spotted were missing yet. Immediately closed Tresorit and fired up my previous DR solution. It took all night to get everything back up to sync and this morning to verify everything before ditching Tresorit completely.

Crisis averted. Still no response from Tresorit despite what now appears increasingly arrogant claims vs. it's competitors in the market. The Linux client came out of beta mid-2016 so should have been rigorously tested.

It's a real shame and I'm extremely disappointed - I really like that they have 2FA across a choice of mechanisms, and claim zero-knowledge across the entire platform (including via the web client - although this is unconfirmed). I'm not concerned with state actors as I've done nothing wrong but I need something that keeps competitors or their agents out - Tresorits Swiss & EU base fits this ideology too. I like the tresor mechanism of sharing and I'm now trying to frig something similar with my DR solution.

Perhaps their Windows and Mac clients are far better at this than their Linux offering so it might be unfair to tarnish their entire platform, but the lack of support (and wasted subscription fees) eroded my trust and after all, what is DR without faith?

Updates

I noticed that Tresorit tweeted a marketing message about feature enhancement, which - to me at least - seemed to confirm that they were essentially ignoring my bug notifications and support requests. I replied to the tweet and the account owner asked me for some details via DM.

After hearing nothing for a few weeks I prompted the Tresorit Twitter account again - this time they promised a response from their support & dev team. Five months to get a response from vendor on a critical issue (and only after complaining on social media).

The explanation given via email was more unsettling than the problem itself - they could not replicate the issue and that some fixes applied since November 2016 'may' have resolved some of the problems. What I take from that is that although none of the fixes were specifically aimed at resolving the problems I reported, they want me to see if they fix it.

In the same email the support member tried to tell me that I must have deleted the files from the tresor on a different machine, which triggered the removal on the machine in question. The problem with that is that I hadn't deleted any of the files on any other machine. The files in question were / are live code files related to an anti-spam module & reporting system I've been designing and writing - there's no way I would delete these files - I've put so much effort in. The other files I found after checking other tresors for mysteriously deleted files were legal documents I would never delete under any circumstances. I rechecked the other machine and I hadn't deleted either set of files there.

Tresorit's support suggested I check the logs - which I did - to see if any files were listed as deleted by user(s). They weren't. Essentially it was just the DR system wrongly flagging files as deleted and that removes all confidence I had using the platform in the first place.

I've since replicated what I liked about the tresor repository system within Spideroak One.

Overall, it's vindication that I made the right decision in abandoning Tresorit altogether, although I'm still wondering what was deleted that I haven't noticed yet.

Wǒ hěn hǎo, xièxie

Some time ago I had a peek into The Other Side and didn't take it any further - but maybe that's because I didn't have a purpose or reason to take it further but I couldn't see a reason to progress, so I didn't. It just came across like a hobbyists environment with a community of snobs driving progress.

Wind the clocks forward another year or more and the landscape is vastly different. I've moved on to learning about network security, information management and have trained myself to think like a black hat (a good defensive strategy). I'm working on some exams that will give me the foundation to absorb that within my work as an architect too and because of the nature of this research I've been working on Linux.

There's some aspects of Windows (e.g. restrictions on packet injection / tampering) which the Linux community seems to lambaste Microsoft for. To me - as a noob at least - it looks like this is by design for commercial reasons. Whatever the reason it just isn't feasible to do a lot of this research on Windows.

So I created VMs through Hyper-V and researched distributions and their capbilities, settling on Debian as my initial preference. It's used as a basis for a number of other flavours including Kali, Raspbian and Ubuntu. KDE is nice and the apt system makes sense to me at this stage.

But then, of course, you start discovering limitations in the virtualised environments leading to one conclusion: You need to deploy to hardware to gain direct interaction with that hardware (and mitigate problems with networking especially). I started beefing up my knowledge of networking stacks and how to analyse network traffic, creating sandbox WiFi networks on my test router and trying to see how to break them / break into them. I found that Kali was a great place to look at this as it contained all the tools and was designed to run OotB so stuck with that on a Pi B+ for a while.

After a while I was using Archimate to design the domains of our house network and started building a HIDS and IDPS, then a DNS server, then spent a bit of cash at ModMyPi getting all the bits I needed. I set up high-grade SSH keys and improved security - I may add a VPN server in the DMZ at some point too. I've got DD-Wrt on the inner router and a custom network set-up which provides additional protection for everyone in the house.

ATX Mid-tower was replaced and needed a new use. Stick a PiRack in there and all the cables.

I suddenly realised I'd become one of the hobbyists I'd turned my nose up years ago. Now our house provides media services so the kids can fire up a film of their choice on any Surface or XBox, iPhone or Windows Phone. We have network protection running in the background emailing me when it detects or fixes a problem. The kids came up with the idea of an underwater camera so they can see the fish even when they hide (yet to be designed and built). None of this involves a Windows server.

Of course I've made significant progress in my learning and research - the next pot of which will be a short study on effective WiFi passwords vs. advice from the pub - but as a by product I've gotten far more technical than I'd expected; you end up finding things to investigate that you'd never considered before and research topics or techniques far from the original purpose.

For example, I've moved my trust away from BitLocker and am testing alternatives, using local accounts for BAU and my Microsoft accounts for connected services (such as OneDrive and XBox). It's not about tin-foil hats, the X-Files or any part of government; it's just a simple case of protecting your assets against criminals or other similar attackers.

I went with Ubuntu because it is Debian-oriented and it seems to have the most support for things that Surface Pro needs. If Debian covered a lot of it I'd have just gone straight there. I don't like the whole Amazon / internet integrations on Unity; the volume buttons don't work; the SP pen buttons don't work; sometimes the left-mouse / pen touch / finger touch just stops responding at random. There's too many suggestions out there on the forums that don't explain what each command suggested actually does (do people just copy and paste these suggestions without understanding the implications first?).

Today is the first time I've used Windows in a week - I love Windows 8.1, especially on Surface Pro. It's beautifully designed, easy to use, makes the switch between keyboard-oriented and tablet seamlessly and OneNote /OneDrive / Office is pure brilliance in design and productivity. LibreOffice and Evolution do Office well but the UX is far clunkier. There is no OneNote outside of Windows and I miss the right click pen button (I only use a mouse on Ubuntu for apps that use context menus a lot). Office365 means I get proper Powerpoint instead of the terrible LibreOffice Impress. There's no Visio equivalent though I'm learning to use Camunda Modeller and Archimate instead. I can operate on client site without Windows now though.

For me Windows 10 is a disaster as it stands. They've ruined OneDrive (where is "Available Off-line Only" for files?) although are promising to rectify the situation and I think they've been led by too many Windows XP-ers in their UI-design-by-community instead of holding their ground and pushing 8.1 on to the next level. Continuum is awesome though - the new W10/Xbox dash is great (game streaming is by far the best add-on here), W10Phone looks superb and I hope they iron out the creases on W10. None of the privacy issues bothered me because you can turn off the telemetry services and disable the data sharing but the OS itself just doesn't feel as coherent or as well thought out as 8.1 on my SP3 or Windows Phone 8.1 on my Lumia.

I'm now in a position where I've had to remove Windows 10 from my replacement Surface Pro 3 as the software licensing service locked the whole machine out (Access Denied); despite this being the default build as supplied by Microsoft. USB boot won't work even after changing the UEFI settings to enable it - I suspect something to do with the Win10 installation - but I've now copied the 8.1 Enterprise installer to a new partition on the SP3 SSD and hacked the Grub2 bootloader to give me the option to boot from it and I'm going to get Windows dual booting on it for OneNote and firmware upgrades. Encrypted SD and data partitions allow sharing between OS-es and decent OpSec can ensure Windows only knows how to access one of those for transfer.

Phew. If you'd suggested and of that to me a year ago my eyes would have glazed over and I would have probably just sent the device back to manufacturer.

But the thing is I'm still afraid to install Win8.1 in case it fudges up all the work done installing and configuring Ubuntu. I know Windows will install its own boot-loader (I've modified the same on my desktop to add back the Ubuntu option enabling dual boot again). I like Ubuntu, Raspbian and Debian - I also like the Windows ecosystem and the journey is never over but I'm reaching the point where I have enough foundation to build on for the security architecture courses. In order to design an architecture or provide solid options for businesses I still feel it's beneficial to understand the inner workings.

It's good to be bilingual between Windows and Linux and none of this has been as difficult as learning Mandarin (as I originally thought it might be). It just sounded more tricky to get started than it was.

Auto-Archiving IMAP in Outlook

It seems like I'm not alone in initially being surprised that IMAP accounts cannot be archived in Outlook.

After spending some time poking around forums, Q&A sites and product support pages it's as simple as IMAP and archiving are mutually exclusive. I thought I'd put a concept forward for anyone out there who needs both the convenience of externally hosted IMAP functionality as well as the maildrop & delivery capability provided by the POP3 system.

A typical example here for me is wanting to access the same email account across multiple devices, get alerts on incoming messages on those devices, and be able to reply should I need to.

I also want to be able to take an archive of older emails (receipts, legal conversations, audit items, records of business and conversations, etc) and store separately for a given period too.

So in order to get around this I use IMAP almost everywhere but then on one (perhaps two locations) I'll connect via POP3 over Outlook - use whatever email client you wish - and use the auto-archive facilities to create email archive files.

Application and Service Relationship

These files (PST) can then be added to an offsite backup. An IMAP account in Outlook will use an OST file to cache mail items and headers but if its deleted or lost your IMAP account is unaffected.

Archive and Artefact Relationships
 

Simple Backup Follow up: Part 2

Ok so having sifted through roadmap candidates I was left with Carbonite, SpiderOak and Backblaze.

As I mentioned in the first part of this piece I've got some very specific [picky] drivers and requirements for this solution.

Carbonite seemed pretty good overall but the price is an issue. For £34 a year (or thereabouts depending on the forex rate) you get to backup only one device. Even the next package up at around £60 a year is restricted to one device.

However for that you get unlimited space on your single Windows or Mac machine. It's not bad but I'm aiming for something that isn't as restrictive to cover my secondary drivers and requirements. To do that I'd have to take one of the Pro Plans, which start at £162 per year. That covers an unlimited number of devices but is then restricted to 250Gb.

It's an option but I'm discounting it for now as I'm going for something cheaper - perhaps even considering Carbonite alongside Datto for an enterprise-level candidate. My concern there is for non-US customers as they have stateside support only according to their website.

So down to two, both of whom have trials available.

I started with Backblaze as it seemed to cover all aspects. The review from the original cloud storage reviews list stated that Backblaze doesn't have a single-point encryption key to match some of the other products but I think the vendor has added the feature since that review.

All fine - good price: Either £3 per month for an essentially unlimited storage quantity, or £9 for the year. I actually thought I need look no further - and for most people this will probably do what you need it to do with minimum hassle. It's pretty easy to use ... but the problem is that I couldn't use it the same way I could with Mozy Pro and define specific backup sets of files and folders. I need a selective DR option and this would take too much time to configure.

With Backblaze I found it would back up all drives, but then allow me to isolate exceptions to the rule to exclude from future backups / delta chains.

Inverse selection....Choose everything then remove everything you don't want

 If it wasn't for that small issue I would have signed up there and then. If you don't have such restrictive requirements and are looking for something safe and cheap you may want to take a look at the options this vendor provides.

My last option was actually added after further research whilst trialling Backblaze, and does exactly what it says on the tin (what I'd call "a Ronseal job").

Whilst the free 2Gb, unlimited devices, hive capable, secure and fast capabilities seem great;  A word of caution: The two-factor authentication is limited as this is a US-focused product too - you cannot use the two-factor authentication unless you have a Canadian or US mobile number. I can get around the problem as I have infrastructure and phone numbers in the states but anyone solely based in Europe would need to review and balance capability over protection.

The vendors engaging the wider FOSS community with outer shell tools and libraries from their product. There's a description of the encryption and hashing algorithms implemented within the web-gumpff pages if you want to read it in detail. Its impossible to tell exactly how they're managing the information protection aspect of the implementation from the sales page but use of CFB is interesting. Works for me.

The only problem I have with that will be future release of open-source libraries used by their main products. Open-source is great but without organisation-level QA of each delta there's a risk of insecurity - lets hope that changes with the major corporate push on critical open source projects from earlier this year. We'll see where that goes but for now I'm going to shortlist SpiderOak.

I've read a few reviews that state that the UI isn't as intuitive; or that its quite complicated - I think thats probably relative. Its more complicated that Backblaze, but probably about the same as MozyPro. The UI is consistent on the Debian package as well so I'll give it a thumbs up.

I like that SpiderOak has endpoint installers for my favourite OS across Windows, Debian-based and Android...but no Windows Phone. We'll see how that goes for now as its not a critical requirement. [Update: WP doesn't need it due to the direct integration with OneDrive]

Whilst chipping away at this article I've been running SpiderOak for a day or so on a selected backup set. I had some problems with the SSL scanner within one of my security suites initially, but have since resolved that issue.

The final candidate, operational across numerous devices.

I ran some tests on a couple of other devices and virtual machines. Windows Server 2012 R2, Kali, Windows 7, Debian and a Mac all worked perfectly well. Time will tell but for now that's all boxes checked. I didn't get round to checking how well it works on the Nexus 7 but there's nothing of value on there anyway. We don't have any overpriced paperweights in this house [c.f. iPad].

SipderOak doesn't store plain text backups, encrypts before transfer and encrypts the transport so prevents easy acquisition of my device files and data.

TL;DR

Overall this is the viable candidate for me, and in summary (comparing it against my original key drivers) I can sync and schedule backups separately, or link the events together - with a per-machine sync schedule. There's a zero-visibility policy meaning only I can unlock the secured backup sets. I can have 2Gb storage free forever - Although I've now signed up to the annual 100Gb package for £60. Its more than I was paying for Mozy Pro but I get more for my money, better support availability and unlimited device capability (including mobile and virtual). I can pick and choose where to restore specific files from any device in my list.

All the candidates I looked at were good products but this one suited my needs better than the rest. I'd be really interested to hear other opinions.

Simple Backup Follow up: Part 1

Having ditched Mozy Pro after trials and tribulations described in an earlier post, I've started looking at alternatives.

I've had no response from MBW or Mozy regarding my password reset or product code requests so couldn't get any further with the uninstall / reinstall process. Needless to say that I haven't got time to spare dealing with the problem, so am looking at other solutions.

Anyone facing a similar choice of offsite backup solutions may find the results useful, but I found this comparison quite a useful starting point. Personally, I'm always a little suspicious of who paid for advertised reviews and which reviews are genuine; so found this list that contained a wide range of solutions.

From my perspective, the term "cloud" is a sales buzzword for architecture that has been in existence for at least a decade. "Cloud", "cloud hybrid", "private cloud" essentially just means "hosted" - With a combination of outsourced hosting or private / internal hosting infrastructure.

Moving past this, the objective of the exercise is to find an offsite / cloud backup solution for personal use - perhaps even a vendor that provides appropriate personal and enterprise-grade solutions. Obviously this is a very specific set of requirements, and yours will be different.

I'm aiming for the following drivers in order:

1. Ability to synchronise and schedule backups, potentially even machine restores
2. Price
3. Security (I'd like a secured backup that only the key-holder can open)
4. Capacity

Optionally, some secondary drivers would be nice:

1. Capable of backing up specific folders / files from a number of devices or VM's
2. Capable of restoring specific files to a device of my choosing

So where to start? Well Mozy Pro is discounted immediately. Whilst it seems to cover the main drivers it seems to miss out on the secondary drivers. Also my own experience has been tainted by the difficulty in solving a problem originally reported in 2010. If I had problems with Windows 8.1 Enterprise I'm not prepared to wait it out or see what happens with Windows 9 upgrades.

After doing some research I'm going to cut the list down to 2 candidates, although I focused on the following roadmap candidates to begin with:

• Carbonite
• Backblaze
• Datto
• OneDrive (Sky Drive)
• SpiderOak

For me, the whole OneDrive / Google Drive / Dropbox mechanism is great for a specific purpose - storing a bunch of files and folders online (or "in the cloud" if you must), and sharing across devices. We have a large proportion of Microsoft devices in our household, along with an iPhone, a few Linux boxes and some other kit I use in my sandbox.

OneDrive is great for allowing the share of files I've acquired on a PC to a sandbox machine on a different VLAN. Its also perfect for being able to capture, modify sales documents written in MS office on Surface Pro, desktops and Windows Phones.

However I've discounted this type of technology almost straight away because I'm looking for a dedicated backup & disaster recovery option for some very specific file sets. Windows 8/8.1 already takes care of things like apps and settings. I've also discounted them because it would be conceivable that MicroGooHoople^TM could allow access (by subpoena, for example) to those backups - don't forget that everything is based in the US your data is liable to US law.

Obviously that last statement is really within tin-foil hat territory :)

I'm also eliminating Datto as it's clearly an enterprise-grade solution (and has no prices on the website!). EtE encryption, Atom 2.4 Ghz 8 core processors on the backup servers, backup chain recovery, bare metal restores, etc.

In part 2 of this post I'll look at the remaining roadmap candidates:

• Carbonite
• Backblaze
• SpiderOak

 So far I'm also seeing encouraging alternatives for all the MBW features I use and will speak to one of the vendors to take the services outside of the MBW package. Great when its all working but appalling when you need assistance.

MY IDE is Better Than YOUR IDE So Nurr

*sigh*
It never really changes.

Same argument - different playground.
jmonkeycoder.wordpress.com/2013/08/28/eclipse-vs-visual-studio/

Interesting article - Think there's plenty of people out there who appreciate the type of comparison.

Thought I'd pitch in as I use both Java and .NET for different clients: I often use both Eclipse and VS (although rarely at the same organisation!). Thought I'd could give a more balanced perspective. You're a Java & Eclipse person - there's nothing wrong with that; many commenters appear to be VS & .NET-ers though and I think there's more caustic discussion there.

It's great to have a side-by-side comparison but there's a lot of the functionality and features from Visual Studio in addition to your lists. There's a number of features from, say Ultimate edition that perhaps not everyone gets to play with! In particular intellitrace and the performance tools from the analysis side, and the architectural tools integration on the design side.

Don't get me wrong - VS doesn't match up to Archimate in my opinion but its tools integrate well. CodeLens is another good example but there's quite a few more.

There are a number Eclipse plugins that do some of these things but not all (which is probably why Ultimate costs > 12k GBP for a single seat license)...but then Java works a little differently, and the platforms it generally runs on are very different!

After running Eclipse on a pretty fast machine on Kali and VS on W8.1 Enterprise I don't see much difference in IDE performance for massive multi-project applications either. I have noticed performance differences in the root frameworks though.

I've always thought that developers shouldn't really have a preference between either framework but I would say there there's a clear difference in the level of productivity however that's kind of irrelevant...I would never consider using Eclipse for .NET/Mono and I haven't seen anything for Java on the Visual Studio side - lets try and forget all about J++ and J# as soon as we all can :)

For me, Java & Eclipse are extremely useful for specific scenarios and I don't think its fair to discount it as other commenters have, yet you've severely underestimated the capability of the other IDE (even for VS 2012).