Ahhh.. Redundancy, a natural stress reliever
Yesterday, May 20th, the server hosting this site had it's operating system drive fail. I never bothered to make the OS drive a mirrored set for various reasons, one of which is because
I figured I had a redundant system in place already. So, I simply had to login to my disaster recovery system and start Apache, and point the DNS to the DR machine. Within a span of 5 minutes the site was back up and functional. This allowed me to casually address the failed drive, a ticket was opened for hands on support to move the drive from slot 2 to slot 1.
Once completed, I just had to setup grub again and the system booted just fine. I address my disaster recovery plan in several ways, one of which is that the actual site is hosted on a XEN virtualized machine. The XEN instances are on a set of software mirrored disks, so the OS disk is strictly the host OS running XEN. When I do major patch updates, I clone the primary disk to the secondary disk for situations such as yesterday. Although the primary server is in California, the backup server is in Colorado, and it also is a XEN virtualized instance.
For the initial setup I just had to clone the XEN intance from California over to the Colorado hosted server. On the primary server I have a series of crons that run periodically to replicate the data over to the backup server using rsync over ssh. The data type determines the frequency of the backup, for instance configuration files that do not change often are performed nightly. Web and databases are replicated every 30 minutes to capture potential changes, such as this post. For DNS and email, I use other providers to manage the redundancy.
For e-mail, I am still using Google for your domain. Sure, some potential privacy concerns, but I treat email as public information unless I encrypt the message. For my DNS, I am using the free DNS offered by NameCheap. So, the only thing I have to worry about, although I must admit I don't lose sleep over it, is my website. I am not even sure if anyone reads this thing, but I still do it for the heck of it.
Some lessons learned:
- Use CNAME's a lot so I do not have to change a ton of DNS records (My site's, my wife's photo sites, etc). I am presently experimenting with something to address this.
- When adding a new XEN system, update the backup drive with the auto start symlink, quick fix.
Really, that is it. It went quite smooth considering I had never actually tested it! This morning I moved the DNS back over to the primary machine. I still need to ship out a new backup drive this week and get the backups of the OS drive going again.
Keep the bits flowing, ScottEducational Video, but will you listen?
For most organizations, it is not a matter of if, but when. In my line of work I have seen some very scary networks, scary enough I would never do business with them again. To be honest, in most cases the smaller organizations are the least secure as they don't want to invest into their network security. Often, in the case of credit card's, these small organizations do not store credit card information. However, what personal information do they have?
Look at your local urgent care shops, the kind that usually have a few PA's, maybe a Doctor that owns the place. Do you believe they invest into their security? Often when you swipe you credit card at these places, it is a terminal device and they do not store this information. However; they have your full name, date of birth, and likely your social security number for working with insurance. All the information anyone would ever need to open credit in your name.
Your credit has significantly more value than your credit card information. Credit card companies have gotten quite good at detecting fraud and stopping it, you will even get your funds returned to you. However, if someone applies for credit in your name, that will cost you a very large sum of money along with time and aggravation. I can not stress enough how locking your credit is very important, you really should do it today!
Back to the enterprise, although many put forth some effort, they do not tend to want to invest in tools that are passive. A proper security organization needs tools to detect and deal with network issues. The idea that a firewall, proxy, and IDS will protect you by themselves is a bit naive. You need to invest in a quality SIEM, you need devices that go beyond signature based detection, you need a proper vulnerability scanning and management system, you need a proper patching process in place, the list goes on. However, many still are stuck in believing a Firewall, IDS, and Proxy is enough to protect them.
For your home, I will have to write a post on best practices to secure your home network as well as one can. Until then, keep your computer patched, remove administrative rights from your account, and switch your home network to use OpenDNS.
Till next time, ScottSarcasm, hmm I should try and control it…
Hmm, I find myself to be sarcastic at times. Is it because I am so full of myself I think others are less intelligent? Or is it just my way of deflecting stress? Perhaps it is my way of telling them they are not what they think they are? I would guess most likely the latter in some cases, in other cases it is just my bad attempt at being funny. I am a computer geek, we are not generally known for our sense of humor.
I have had the opportunities to work with people who try and talk a big game and think they are more important than they are, I find sarcasm comes out around these types. I want to tell them I can replace you, we are all replaceable in the working world. There is not one thing that only one or two people know how to do. I believe everyone has the intelligence to learn anything they want to commit themselves to. So, don't think you can't be replaced and you will be humbled once you realize it.
You certainly want to figure this out early, as I have released people based on their pretentious attitude towards others and their drive to horde information. Guess what? They realized when we never called them that life goes on and we replaced them just fine. We like team players in this world, not pretentious asses. With that said, I believe I currently work in a place where everyone appears to be down to earth and humble.
Looking back, many of those who displayed this type of pretentiousness were most likely over compensating for where they knew they were lacking. The problem with this idea is that you will eventually be found out. Don't be afraid to admit you don't know, I don't expect everyone to know everything, but being motivated to figure it out certainly has character. I truly believe most people, especially manager types, are like this. If not, you need to find a better person to work for as they are likely a pretentious ass.
In this post, I realize I need to find a different word than pretentious, I have used it a lot. But, that is okay, it gets the point across. Moral of the story, don't think you know more than the next guy, and be open to opinions of others. Your way is not necessarily the correct way, and me asking for details or giving other idea's is not an insult to your intelligence. I am trying to create constructive dialog to help develop the best plan or solution as a team, in the end we may very well go with your idea.
Until next time, keep the bits protected! ScottOpenDNS, safety in numbers.
Just a quick overview of the Domain Name System (DNS); it is the system that takes Google.com and turns it into an Internet Protocol (IP) address such as 74.125.225.166, which is what your computer actually uses to reach out to the web server and retrieve the content. Think of it as the mailing address of the Internet. With that said, DNS see's everything you go to or send via the Internet. Of course in most cases your Internet Service Provider (ISP) provides the DNS services for you. Generally they don't care where you go, they just look up the IP for you and pass it along.
However, if you use the right DNS provider, you can add protection to your network. This is where OpenDNS comes in! By using OpenDNS you get basic malware and phishing protection, which everyone could use. If you opt to create an account, you can register your IP address of your home router and you can add additional filtering. I use it for my home network to block anything we have no need or desire to go to. This is done to protect my kids from inadvertently clicking on a link and viewing something they shouldn't, such as adult content.
If you have a dynamic IP, no worries, they have answers for that too! Most home users are on a dynamic IP address, so you will want to look at their directions for working with this. Once you register your IP address with OpenDNS, you can filter all sorts of categories, so take a look and see what they have available. Granted this will be a home wide solution, so you will not be able to block social media from your kids if you want to use it for yourself. There are other options for this, many can be complicated for the non-technical user.
For a more individual computer level of protection, take a look at K9 Web Protection. Although you should still use OpenDNS to protect your entire home network, K9 will allow you to block sites on your kids computers that you still want to be able to access.
So much for privacy settings on Facebook!
If you have any interest in personal privacy, I recommend you drop social media completely.
With the recent addition to Facebook called "graph search" privacy settings are pretty much useless. Although Facebook removed the ability to prevent yourself from being searchable previously, things like pictures and content were still safe. Now, with graph search, this is no longer the case. And it would appear the more likes you get the worse off you are. Of course deactivating your account does not prevent this, you must delete your account.
This is something that is a little more obscure, but if you go to StartPage and search "delete Facebook account", you will end up with a link to point you in the right direction. Originally I had deactivated my account, so I had to reactivate in order to request deletion. Interestingly enough it is said to take 14 days to delete your account, probably so they can make sure they index and archive all your data for their current and future nefarious activities. If you have an interest in keeping your data, you can download a backup of your content prior to requesting deletion.
Not to try and scare anyone, but think about the information that is necessary to open credit in your name. Now think about how much of that information is publicly available now, much of it put out there by you. Now think about going forward as organizations such as Facebook and Google want to index more and more of your personal data. I have spoken about changing your search engine and how to protect your credit, it is time people actually care about their privacy and change how they do business on the Internet.

