I have been making a lot of changes the network recently so thought it was time to sketch it out:
The left hand stack is housed in the A/V rack/closet and the right stack is housed in the garage rack. To clarify a few points:
- The two switches are linked using a 2 x 1 gigabit fibre trunk
- The ESX servers and file servers are connected to the switch using 2 x 1 gigabit ethernet trunks
- There ET9000, XBMC, Apple TV 2, HDMI Matrix and Sonos devices can all be controlled from smartphones, pcs etc
- The ET9000 records to the file server
- The Sonos Connect / ZonePlayer 90 is connected to the Onkyo Amp to allow Sonos to output to my home theatre
- The Sonos Connect / ZonePlayer 90 is connected to the Apple TV to allow streaming from iTunes/iOS devices to all zones
- I haven’t bothered to include my router, wireless access points, client pcs, smartphones, printers and various other devices in the diagram
The wireless sensor network provides:
- Whole house power consumption currently using a CurrentCost EnviR
- Individual appliance power consumption (currently monitoring 6 appliances using CurrentCost IAMs)
- Environment information (temperature, light, humidity) using Jeelabs Jeenodes in multiple zones
This is a good reference point for where I’m headed (not much progress since I last listed but alas, still the way forward):
- Further appliance monitoring and more importantly control (potentially through upcoming project/device: HexaBus – https://github.com/mysmartgrid/hexabus/wiki or plugwise – http://www.plugwise.com/idplugtype-g/)
- Security cameras (and doorbell integration)
- Environmental information (I have pulse counters for my gas meter and water/central heating system that need installing)
- XBMC hardware replacement (once the Raspberry Pi – http://www.raspberrypi.org/ is available to buy I’d like to replace my Acer Revo)
- Home Automation Hub replacement (once the Raspberry Pi is available to buy I’d like to try Domitiga – http://www.domotiga.nl/ or my own custom solution)
- Web Interface (to bring all of the information together from the wireless sensor network, along with the state of all network devices (from bandwidth to free space etc) and most importantly allow central control- so instead of me having to use different methods to switch on the ET9000, change the channel on the HDMI matrix, power on the amp etc, I can do it all with one button)
As usual- I imagine I’ve missed a few bits out from the above but it hopefully provides a good overview of my network.
Wireless network not operating at 802.11n 130 Mbps speed (only running at 802.11g 54 Mbps): I won’t go into too much detail- but the resolution was to change the security from WPA to WPA2 (seems crazy to me that the web interface doesn’t make this incompatibility clear).
ESXi / NFS problems: My virtual machines kept dieing and there were a lot of NFS events in vSphere "Lost connection to server x mount point y mounted as z". This must be related to the network maintenance over the last few weeks but I’m not quite sure why. I ended up fitting the new HP NC360T dual-port gigabit NICs to both HP Microservers then migrating the datastore to FreeNAS. Since then *fingers crossed, touch wood etc* I’ve had no problems!
I want to talk a bit about LACP but that’ll have to wait for another time:
A few small updates as I have a ton of projects on the go at the moment.
After a friend visited earlier in the week I decided to give FreeNas / ZFS another shot- so I’ve installed the latest v8 on a spare HP Microserver N36L. The little cube has 4x 1.5TB Samsung disks in RAID-Z with a single 4GB stick of RAM. I have ordered another stick of RAM and a low profile, dual-port, gigabit pci-e network interface card (nic)- the HP NC360T (intel chipset) so I can setup load-balancing/link aggregation/LACP. I starting copying some data off of the current production Microserver (WHS2011):
Performance seems average. I’d expect to see a bit of an improvement when the gigabit HP / Intel nics arrive.
New Orders :)
I have put my Cisco SGE2000P 24-Port Gigabit managed PoE switch up on eBay as it’s a bit too big and loud for the small data/comms cabinet I bought recently. As a replacement I have ordered the D-Link DGS-1224TP which appears to offer the same feature set at half the size and half the weight (I think it may also be passively cooled and is touted as being "green" so hopefully help with the leccy bill :)
I also have a few Sonos S5 network wireless hifis on order (they were on special offer and I’ve been looking at whole house / multi-room audio distribution solutions for a while). I have a slight concern I may need to buy some Apple AirPort Express access points to enable AirPlay on them but I’ll cross that bridge as/when…
I’m learning xcode / cocoa / objective-c / osx / iOS whatever they call it as I want to write a little app to control my 4×4 HDMI matrix. The device is on the network and supports simple commands using a telnet style socket connection, so I’m hoping it won’t be too tricky! Unfortunately I couldn’t find the old control software I had or documentation but eventually stumbled upon the pdf with a few pointers. So the reading begins… http://developer.apple.com/library/mac/#referencelibrary/GettingStarted/Learning_Objective-C_A_Primer/_index.html#//apple_ref/doc/uid/TP40007594
I’ve decided to stick with my rackable systems box as my fileserver because it’s relatively quiet and “green”. I have two identical servers so all i had to do was pop 4x 1.5TB drives in the new one (the old one had 4x 750GB) and migrate the data! The old server is running FreeNAS 7.2 and I loaded FreeNAS 8 RC2 onto the new one (ZFS support).
I can briefly remember attempting the migration task in the past and finding it somewhat easier said than done!
First i tried from a windows box connecting to each server using samba simply dragging/dropping the files. The transfer goes from the old server to the windows box then back to the other server so I decided this was a no go (performance was pretty poor and the samba process on both servers seemed to be eating the CPU but at least i got a progress bar).
Now I decided if i want the transfer to go directly from server to server i’d need to SSH into one of them and initiate the transfer from there. I mounted the old server on the new (mkdir /mnt/oldnas | mount oldnas:/mnt/data /mnt/oldnas) and proceeded to copy a folder (cp -R /mnt/oldnas/test /mnt/data). Unfortunately I couldn’t tell how fast the transfer was going nor the progress it had made!
So googling suggested rsync (rsync –progress) which provided speed and progress- great! Or not… unfortunately performance was even worse than before, only this time rsync was eating the CPU!
Next google result… scp (scp /mnt/oldnas/test newnas:/mnt/data)- using both local paths doesn’t display progress so one path must be nfs. Poor performance again! You guessed it… scp eating CPU.
So… back to cp! This time i passed -v to get an update after every file completes (cp -R -n -v /mnt/oldnas/test /mnt/data). Excellent! Only 10% CPU utilization… but can i determine how fast the transfer is going without timing how long a file (or group of files) takes to transfer? After trying a bunch of useless commands i finally found systat -ifs which shows current network throughput and peak network throughput- perfect!
Another update shortly!
P.S. I found that: systat -ifs doesn’t work prior to FreeBSD 8 (not a problem for me as i think FreeNAS 8 RC2 is based on FreeBSD 8.2) but: netstat -Iem0 -w1 -h is mildy helpful if you are running FreeBSD < 8.
My “new fileserver” project had been on hold for a bit what with moving house etc but now i’m moved i thought i’d give it another go!
The “brick wall” i’d previously hit was my supermicro sata expansion cards (AOC-SAT2-MV8) were being picked up by the highpoint rocket raid driver (hptrr) when in fact they should be handled by a different driver. After some searching i found information suggesting a full FreeNAS install would allow me to tweak the /boot/defaults/loader.conf (adding hptrr_load=”NO”) to get around this problem.
Unfortunately- that didn’t work! Some more research suggested i needed to recompile the FreeNAS kernel without the hptrr driver, but this is really a bit beyond my expertise! So i kept searching!
Finally, i found another /boot/defaults/loader.conf suggestion (adding hw.hptrr.attach_generic=0) and voila- i was up and running!
So i added my disks (Disks|Management) and created my virtual device (Disks|ZFS|Pools|Virtual device), but when i add a pool and click apply i get an error: The changes could not be applied (error code 1). And in the log: cannot create zpool permission denied.
A final bit of googling reveals a command: sysctl -w kern.geom.debugflags=17 – finally i’m able to create my zpool :)
That’s all for now- i need to migrate some more of my network equipment before i can begin benchmarking etc.
Damn- i’ve already got lazy and not been updating regularly as i told myself i would. A few bits i remember since last time…
-New fileserver arrived and i have now realised the 2 raid cards i have are useless (i need SATA non-raid cards) for ZFS with FreeNAS. Currently hunting on eBay for something else!
-My rack tft/lcd keyboard/mouse thing packed up- so i need to look for a new one of those too!
-LINQ to SQL: I got hooked a while back but didn’t like the way it interprets some manipulation and couldn’t cope with bulk updates. But now i figured i can simply do db.executecommand() i’m happy!
-I got trixbox working nicely with the Cisco Wireless 7920 IP Phone i bagged cheap off eBay (using SCCP):
setup static ip/dns (system-config-network)
create some extensions?
yum install asterisk16-devel gcc
wget Chan_SCCP V3.tar.gz
tar -zxvf Chan_SCCP…
copy sccp.conf (from /root/Chan… to /etc/asterisk) + edit
edit /etc/asterisk/modules.conf (add:
noload => skinny_sccp.so
load => chan_sccp.so
-Communicating between multiple forms in vb.NET: I was struggling to figure how to pass information to another form without storing it in a control. I ended up doing something like:
Public Class Form2
Dim edit_acct_row as Int32
Public Sub New(ByVal row_index As Int32)
edit_acct_row = row_index
then calling the form like:
Dim f_pick_acct = New Form2(edit_acct_row)
-Checking/Ticking a checkbox in a DataGridView with vb.NET: When simply manipulating the dgv.rows(r).cells(c).value the state of the checkbox was not updating! I ended up having to:
Form1.dgv_preview.Rows(r).Cells(c).Value = True
-My Simpsons Kesha (Ke$ha) Tick Tock Intro Video on youtube has had over 10,000 hits! I’m not working on another video which will take quite some time but i hope will attract quite a lot of attention once complete :)
Wow- a lot squeezed into a small space + tons i’m sure i’ve learnt and forgotten!
Some of the fiber gear has arrived: (cables and, what i thought were NICs but are in fact HBAs- all part of the learning process!)
Not really much use till i get the fiber switch and new fileserver (i think i’ve picked the hardware which i’m going to get shipped from the US- still a lot cheaper than sourcing here in the UK). No idea for the operating system- whether to go with Windows Home Server, Freenas, Openfiler or something else? ZFS or hardware raid?
I’m also in the process of moving house, so all my gear will get re-racked then i can hopefully post some shots up!
P.S. Is it fiber or fibre?
P.P.S. I possibly saw the best Simpsons intro ever this afternoon so i’m going to try and cap it and post it up (there’s a few back-to-front poor quality versions on youtube…)