HomeLab – Upgrade Main Storage Server from TrueNAS Core to TrueNAS Scale May Not Be a Good Idea

After waiting for a few months, I finally upgraded my main storage server from TureNAS core to TureNAS Scale during my server upgrade. Now I am asking myself, do I really need this upgrade?

Homelab – Basic VM Disk IO Benchmark

I have had a storage server upgrade last few weeks, and just did some IO benchmark in my hypervisor.

I am using VMware ESXi as my hypervisor on a dedicated machine. it hooks on my Truenas Scale storage server with a 10G iSCSI connection and standard 1500 MTU (no jumble frame.)

Tests are performed with fio, and Google provides a simple guideline for VM disk IO benchmark https://cloud.google.com/compute/docs/disks/benchmarking-pd-performance

Test write throughput by performing sequential writes with multiple parallel streams (8+), using an I/O block size of 1 MB and an I/O depth of at least 64:

sudo fio --name=write_throughput --directory=$TEST_DIR --numjobs=8 \--size=10G --time_based --runtime=60s --ramp_time=2s --ioengine=libaio \--direct=1 --verify=0 --bs=1M --iodepth=64 --rw=write \--group_reporting=1
# sudo fio --name=write_throughput --directory=$TEST_DIR --numjobs=8 \--size=10G --time_based --runtime=60s --ramp_time=2s --ioengine=libaio \--direct=1 --verify=0 --bs=1M --iodepth=64 --rw=write \--group_reporting=1
write_throughput: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.19
Starting 8 processes
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
Jobs: 7 (f=7): [W(4),_(1),W(3)][52.5%][w=64.9MiB/s][w=64 IOPS][eta 00m:58s]
write_throughput: (groupid=0, jobs=8): err= 0: pid=2502: Thu Jun 16 17:30:06 2022
  write: IOPS=224, BW=229MiB/s (240MB/s)(13.7GiB/61466msec); 0 zone resets
    slat (usec): min=47, max=1918.2k, avg=22289.12, stdev=138620.30
    clat (msec): min=19, max=49855, avg=1843.07, stdev=5712.01
     lat (msec): min=176, max=50052, avg=1865.65, stdev=5811.18
    clat percentiles (msec):
     |  1.00th=[  180],  5.00th=[  184], 10.00th=[  300], 20.00th=[  735],
     | 30.00th=[  776], 40.00th=[  793], 50.00th=[  827], 60.00th=[  860],
     | 70.00th=[  919], 80.00th=[ 1301], 90.00th=[ 1569], 95.00th=[ 2072],
     | 99.00th=[17113], 99.50th=[17113], 99.90th=[17113], 99.95th=[17113],
     | 99.99th=[17113]
   bw (  KiB/s): min=28617, max=1160657, per=100.00%, avg=252403.65, stdev=41662.93, samples=472
   iops        : min=   21, max= 1129, avg=240.85, stdev=40.74, samples=472
  lat (msec)   : 20=0.01%, 100=0.01%, 250=9.83%, 500=0.87%, 750=13.68%
  lat (msec)   : 1000=53.79%, 2000=18.23%, >=2000=5.37%
  cpu          : usr=0.20%, sys=0.38%, ctx=13534, majf=0, minf=467
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.1%, 16=0.6%, 32=1.2%, >=64=98.1%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,13827,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=229MiB/s (240MB/s), 229MiB/s-229MiB/s (240MB/s-240MB/s), io=13.7GiB (14.8GB), run=61466-61466msec

Disk stats (read/write):
    dm-0: ios=0/16134, merge=0/0, ticks=0/12396824, in_queue=12396824, util=99.90%, aggrios=0/16134, aggrmerge=0/1, aggrticks=0/12424846, aggrin_queue=12424845, aggrutil=99.87%
  sda: ios=0/16134, merge=0/1, ticks=0/12424846, in_queue=12424845, util=99.87%
[root@localhost Data]#

Test write IOPS by performing random writes, using an I/O block size of 4 KB and an I/O depth of at least 64:

sudo fio --name=write_iops --directory=$TEST_DIR --size=10G \--time_based --runtime=60s --ramp_time=2s --ioengine=libaio --direct=1 \--verify=0 --bs=4K --iodepth=64 --rw=randwrite --group_reporting=1
[root@io-test ~]# clear
[root@io-test ~]# sudo fio --name=write_iops --directory=$TEST_DIR --size=10G \
> --time_based --runtime=60s --ramp_time=2s --ioengine=libaio --direct=1 \
> --verify=0 --bs=4K --iodepth=64 --rw=randwrite --group_reporting=1
write_iops: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.19
Starting 1 process
fio: io_u error on file /Data/fiotest/write_iops.0.0: No space left on device: write offset=303255552, buflen=4096
fio: pid=2596, err=28/file:io_u.c:1803, func=io_u error, error=No space left on device

write_iops: (groupid=0, jobs=1): err=28 (file:io_u.c:1803, func=io_u error, error=No space left on device): pid=2596: Thu Jun 16 17:32:46 2022
  write: IOPS=60.0k, BW=238MiB/s (250MB/s)(5709MiB/23972msec); 0 zone resets
    slat (usec): min=3, max=36791, avg= 8.45, stdev=39.28
    clat (usec): min=184, max=177493, avg=1040.16, stdev=2205.25
     lat (usec): min=198, max=177499, avg=1048.82, stdev=2206.88
    clat percentiles (usec):
     |  1.00th=[   635],  5.00th=[   717], 10.00th=[   750], 20.00th=[   799],
     | 30.00th=[   832], 40.00th=[   857], 50.00th=[   889], 60.00th=[   922],
     | 70.00th=[   963], 80.00th=[  1020], 90.00th=[  1139], 95.00th=[  1647],
     | 99.00th=[  3294], 99.50th=[  4686], 99.90th=[  9765], 99.95th=[ 15008],
     | 99.99th=[160433]
   bw (  KiB/s): min=82464, max=294088, per=100.00%, avg=245998.02, stdev=67413.87, samples=47
   iops        : min=20616, max=73522, avg=61499.38, stdev=16853.44, samples=47
  lat (usec)   : 250=0.01%, 500=0.01%, 750=9.57%, 1000=67.11%
  lat (msec)   : 2=18.91%, 4=3.75%, 10=0.55%, 20=0.05%, 50=0.02%
  lat (msec)   : 100=0.01%, 250=0.02%
  cpu          : usr=14.15%, sys=45.55%, ctx=156240, majf=0, minf=68
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,1461581,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=238MiB/s (250MB/s), 238MiB/s-238MiB/s (250MB/s-250MB/s), io=5709MiB (5987MB), run=23972-23972msec

Disk stats (read/write):
    dm-0: ios=0/1592185, merge=0/0, ticks=0/1429699, in_queue=1429699, util=99.56%, aggrios=0/1591823, aggrmerge=0/362, aggrticks=0/1422162, aggrin_queue=1422162, aggrutil=99.56%
  sda: ios=0/1591823, merge=0/362, ticks=0/1422162, in_queue=1422162, util=99.56%
[root@io-test ~]#

Test read throughput by performing sequential reads with multiple parallel streams (8+), using an I/O block size of 1 MB and an I/O depth of at least 64:

sudo fio --name=read_throughput --directory=$TEST_DIR --numjobs=8 \
--size=10G --time_based --runtime=60s --ramp_time=2s --ioengine=libaio \
--direct=1 --verify=0 --bs=1M --iodepth=64 --rw=read \
--group_reporting=1
[root@io-test ~]# sudo fio --name=read_throughput --directory=$TEST_DIR --numjobs=8 \
> --size=10G --time_based --runtime=60s --ramp_time=2s --ioengine=libaio \
> --direct=1 --verify=0 --bs=1M --iodepth=64 --rw=read \
> --group_reporting=1
read_throughput: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.19
Starting 8 processes
Jobs: 2 (f=2): [_(1),R(2),_(5)][100.0%][r=14.0GiB/s][r=15.3k IOPS][eta 00m:00s]
read_throughput: (groupid=0, jobs=8): err= 0: pid=1881: Thu Jun 16 17:48:29 2022
  read: IOPS=21.4k, BW=20.9GiB/s (22.4GB/s)(1259GiB/60292msec)
    slat (usec): min=13, max=76074, avg=197.95, stdev=422.19
    clat (nsec): min=1404, max=693868k, avg=23722148.51, stdev=59989033.56
     lat (usec): min=86, max=693925, avg=23920.38, stdev=60025.44
    clat percentiles (msec):
     |  1.00th=[   12],  5.00th=[   12], 10.00th=[   12], 20.00th=[   12],
     | 30.00th=[   12], 40.00th=[   13], 50.00th=[   13], 60.00th=[   13],
     | 70.00th=[   13], 80.00th=[   13], 90.00th=[   13], 95.00th=[   14],
     | 99.00th=[  342], 99.50th=[  347], 99.90th=[  384], 99.95th=[  414],
     | 99.99th=[  456]
   bw (  MiB/s): min=18584, max=26532, per=100.00%, avg=21483.09, stdev=176.33, samples=954
   iops        : min=18579, max=26529, avg=21478.41, stdev=176.35, samples=954
  lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 100=0.01%, 250=0.01%
  lat (usec)   : 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.07%, 20=96.31%, 50=0.02%
  lat (msec)   : 250=0.01%, 500=3.57%, 750=0.01%
  cpu          : usr=0.82%, sys=49.54%, ctx=41872, majf=0, minf=466
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=1288207,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=20.9GiB/s (22.4GB/s), 20.9GiB/s-20.9GiB/s (22.4GB/s-22.4GB/s), io=1259GiB (1351GB), run=60292-60292msec

Disk stats (read/write):
    dm-0: ios=47642/87, merge=0/0, ticks=15825386/36345, in_queue=15861731, util=100.00%, aggrios=47652/75, aggrmerge=1189/12, aggrticks=15817689/23298, aggrin_queue=15840987, aggrutil=100.00%
  sda: ios=47652/75, merge=1189/12, ticks=15817689/23298, in_queue=15840987, util=100.00%
[root@io-test ~]#

Test read IOPS by performing random reads, using an I/O block size of 4 KB and an I/O depth of at least 64:

sudo fio --name=read_iops --directory=$TEST_DIR --size=10G \
--time_based --runtime=60s --ramp_time=2s --ioengine=libaio --direct=1 \
--verify=0 --bs=4K --iodepth=64 --rw=randread --group_reporting=1
[root@io-test ~]# sudo fio --name=read_iops --directory=$TEST_DIR --size=10G \
> --time_based --runtime=60s --ramp_time=2s --ioengine=libaio --direct=1 \
> --verify=0 --bs=4K --iodepth=64 --rw=randread --group_reporting=1
read_iops: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.19
Starting 1 process
read_iops: Laying out IO file (1 file / 10240MiB)
fio: ENOSPC on laying out file, stopping
Jobs: 1 (f=1): [r(1)][100.0%][r=1061MiB/s][r=272k IOPS][eta 00m:00s]
read_iops: (groupid=0, jobs=1): err= 0: pid=1913: Thu Jun 16 17:50:33 2022
  read: IOPS=271k, BW=1057MiB/s (1108MB/s)(61.9GiB/60001msec)
    slat (nsec): min=1569, max=697200, avg=2087.86, stdev=1413.17
    clat (usec): min=2, max=11769, avg=233.84, stdev=26.11
     lat (usec): min=5, max=11886, avg=236.04, stdev=26.43
    clat percentiles (usec):
     |  1.00th=[  182],  5.00th=[  204], 10.00th=[  217], 20.00th=[  225],
     | 30.00th=[  225], 40.00th=[  227], 50.00th=[  229], 60.00th=[  233],
     | 70.00th=[  237], 80.00th=[  243], 90.00th=[  260], 95.00th=[  273],
     | 99.00th=[  330], 99.50th=[  359], 99.90th=[  441], 99.95th=[  562],
     | 99.99th=[  816]
   bw (  MiB/s): min=  946, max= 1085, per=100.00%, avg=1058.05, stdev=22.12, samples=119
   iops        : min=242382, max=277826, avg=270859.52, stdev=5662.33, samples=119
  lat (usec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  lat (usec)   : 250=85.50%, 500=14.43%, 750=0.05%, 1000=0.02%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=48.33%, sys=51.25%, ctx=281, majf=0, minf=58
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=16234920,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=1057MiB/s (1108MB/s), 1057MiB/s-1057MiB/s (1108MB/s-1108MB/s), io=61.9GiB (66.5GB), run=60001-60001msec

Disk stats (read/write):
    dm-0: ios=2831/16, merge=0/0, ticks=1340/15, in_queue=1355, util=6.47%, aggrios=2833/10, aggrmerge=0/6, aggrticks=1338/9, aggrin_queue=1346, aggrutil=6.45%
  sda: ios=2833/10, merge=0/6, ticks=1338/9, in_queue=1346, util=6.45%
[root@io-test ~]#

Tutorial 1: Creating CentOS Base-Image for Atlassian Applications

In this tutorial, we are going to creating a CentOS 7 base image for our Atlassian applications.

In our cases, the base image is will be created on an ESXi based virtual machine. If you are using other hypervisor or running on a bare-metal you can also follow this tutorial.

If you are using a cloud provider such as AWS, Google Cloud, DigitalOcean, the base image we are creating is very similar to their image.

Homelab – My Third SUPERMICRO SC846 Chassis

Today I get another Supermicro SC846 4U Chassis for my Homelab.

Two months ago, one my friend wanted a dual-CPU motherboard for an old workstation setup and asking for my opinion. I replied if you need a second-hand server board it’s better to buy a barebone with a chassis to avoid repaired parts. So we bought a chassis with a Supermicro X9DRI-LN4F+ motherboard delivery to my friend’s place, and he keeps the motherboard, and I take the chassis. The system arrived in one week, but I was not going to take my chassis due to Coronavirus.

Finally today I clean up some space on my rack and pick up this 20kg metal box from my friend’s storage.

This chassis is a Supermicro SC846 4U chassis OEM for Inspur – Chinese server manufacture, as the I bought this server in China. Besides there are several Chinese stickers on the outside of the box and Supermicro logo is replaced with Inspur logo on right server handle, there are no different to any Supermicro chassis. There are Supermicro marker and part number everywhere inside.

Chassis
The first look is the chassis is in excellent conditions. The top of HDD cage, chassis door and chassis handle was protected by original plastic film, all surface is shine without rust or dust. Beside of reasonable marks, there is one visible dent on top of HDD cage.

Fans
The recent model of CSE-846 may come with three Nidec 7000 rpm fan (FAN-0127L4) with 72.5 CFM and 53.5 dBA in the middle and two Sanyo Denki 6700 rpm fan (FAN-0125L4) with 59.6 CFM and 47 dBA at the exhaust, but my configuration have five Sanyo San Ace 80 6700 rpm fans. Those fans are very loud at any configurable speed and not suitable in any home office environment, so I will replace them with much quieter Noctua fans.


Backplane
My chassis comes with a SAS846EL1 backplane, which is a very old SAS 1 backplane with only 12Gbit/s (4 lines x 3 Gbit/s) uplink to the controller, which will be probably replaced by direct plane or SAS 2 plane. I have summarised all backplane for SC846 chassis in this post.

Drive Tray
There are 24 Gen 5.5 hot-swap 3.5 hard drive trays (MCP-220-00075-0B) come with this chassis. As always Supermicro drive tray is ugly, I don’t like the wave design, and the small tray handle makes it difficult to pull drive out. The Gen 6.5 tray (MCP-220-00094-0B) has a much better look and handle design, but cost a fortune (RRP US$ 887.76) to replace all of my 72 trays and additional time to screw all the hard drives.

On the other hand, compared to most HP and DELL tray, most of Supermicro drive tray has much large hole, allow more airflow to go through the hard drive. Also, all of them includes plastic dummy hard drive, which allows airflow to go through the hard drive and preventing airflow bypass from the empty drive bay.

Lucky the chassis includes an internal drive tray (MCP-220-84603-0N), it is expensive (cost US$ 29.99 brand new), and very hard to find in China.

Power Supply
Chassis support redundant power supply, but only one 900W power supply (PWS-902-1R) come with the package, it’s load and not 80 Plus certified. All of my chassis come with this type of PSU, but I replace with much quieter 920W PSU with SQ (PWS-920P-SQ).

On Rack
Installed with 4U Rail Kit (MCP-290-00057-0N) on the rack. The Supermicro rail looks thin, but very well made, easy to install (no tool or stupid cage nuts required). It’s a ball-bearing rail, I can easily pull a full load server all the way out.

Finally, looks great with other servers. I left 1U between server for better cooling, and avoid a collision when I pull out the server.

Homelab – Backplane for SUPERMICRO SC846 Chassis, the Buying Guide

Title Photo by Magnus Engø on Unsplash

Supermicro SC846 is a great chassis for homelab users, which offering 24 3.5inch hard drive bay with varieties of backplanes. Most users buy second-hand chassis and want to change backplane because it doesn’t fit their needs. I list all backplanes for SC846 until April 2020 and also my recommendations for the different use cases.

Homelab – How to Organise Equipment

Title Photo by Magnus Engø on Unsplash

Group your equipment together

Group your equipment together, make sure you can access most of your equipment at the same location at the same time. This will reduce lots of effort when tracing an unknown device issue – you don’t have to run between different places.

You can have your equipment group in two or three locations, but you should not put them everywhere at home, except some special network device such as WiFi access point, or communication device, for example, VoIP phone.

Put in a fixed location

The system should be located on a fixed location at home. Depends on the country you located, house structure you stay and types of equipment you have, the location of your home lab can be different.

Choose good locations is one of the most challenge part of your home lab setup. there are many variables have to consider

Power accessibility

First of all, there must be a power source close to your homelab device. it is not a good idea of running long extension power cable without a certified electrician. Using extension power cable with high power equipment can be a fire hazard with, and not secured cable can cause accidents.

Provide dedicated circuit and power lets for your equipment if possible. Do not share a circuit with other high power electrical appliances such as air-conditioning, electrical oven or electrical heater. You don’t want your server lost power because of an overcurrent or short circuit in a different room far from your homelab.

Network accessibility

Make sure you have enough existing network cable access to your homelab location. You should also have a backup line for your core network in case of a cable problem.

Evaluate your cable for future upgrade, for example, it will be very difficult to run a 10G network on a CAT5e cable. Consider the difference between a copper-based network or a fibre-based network, and which type of network your device supports.

Server noise

Many enterprise equipments are designed for running inside a server room or datacenter, in most of the cases server noise is not a concern. Several new server models from HP and DELL is much quieter than before, but running noise may sill too loud for most of the people, especially at night.

Thermal control and ventilation

Homelab equipment consumes electric and produces heat. Now new server cloud operates under higher environment temperature such as 28 Celsius, but some parts such as hard-drive may lose service life.

Heat can also cause server fan running at very high speed, and cause extra noise. We are human, we can not work in an extremely hot environment.

Working space and accessibility

You should keep enough workspace for your homelab and make sure all equipment is easy to access. This should including enough spaces when not maintain your server and when you are moving your equipment.

Most servers are flat and long, they may difficult to go through narrow corner or doors. A full hight server cabinet can easily over 2 meter high, which makes them not fit in some basement or attic.

Servers are generally made by steal, which means they are heavy, very heavy. An empty 2U DELL or HP rack server can be easily over 20+ kg without any hard drive installed. Fully loaded 4U storage boxes can be heavy as 50+ kg, and you need at least 2 person to move those boxes, or using lift equipment. so make sure you still have space for your friends and carry equipment for your server.

A fully loaded 42U server cabinet can be heavy as 2000 kg, with footage less than 2 square meters. make sure your floor support a heavy load, and not make holes on it.

The place for your homelab

There are some commonplace for your homelab. they have different pros and cons.

Dedicate server room

Pros: This could be the best place for your home lab.

  • Easy to access because inside of your house or apartment, and providing a better work environment.
  • Generally, quilt if the room is surrounded with a thick wall or using soundproof material.
  • Generally, cool if veneration equipment is properly installed and used.
  • Blend in, hide your generally ugly server cabinet.
  • Dedicate room reduce risk from accident damage and natural hazard such as flood, heatwave, bugs, etc… Also void damage from and to kids and pets.

Cons: This could be the most expensive plan.

  • Using precious space of your house or apartment.
  • High cost for renovation from the existing room, and equipment installation.
  • Limited workspace if the room is small.

Study room or home office room

Pros:

  • Usually enough space to work.
  • Acceptable environment temperature.
  • Close to the working desk or your development space. Easy to access because inside of your house or apartment.
  • Generally better network cable set up in the study room or home office room.

Cons:

  • Noise is an issue.
  • Could be ugly.

Living room

Pros:

  • Usually lots of working space.
  • Acceptable environment temperature.
  • Easy to access because inside of your house or apartment.
  • Show off, every visitor knows you have a big rack.

Cons:

  • Could be ugly.
  • Noise is a major issue.
  • You wife (or husband) may hate you.

Garage

Pros:

  • Generally quilt, noise is not a problem in the garage.
  • Enough space for work, garage is a large space.
  • Hide your generally ugly server cabinet.

Cons:

  • Running network cable to other room can be difficult.
  • Can be far from your working desk, may require lots of walking.
  • May not have animal control. Be careful of bugs, cat, mouse, etc..

Basement

Pros:

  • Generally quilt, noise is not a problem in the basement.
  • Enough space for work, the basement is a large space.
  • Hide your generally ugly server cabinet.
  • Generally, the temperature is low in the basement.

Cons:

  • Risk of flood and water damage.
  • Access to the basement can be difficult, and moving equipment to the basement may also difficult.
  • Can be far from your working desk, may require lots of walking.
  • May not have animal control. Be careful of bugs, cat, mouse, etc..

Attic

Pros:

  • Generally quilt, noise is not a problem in the attic.
  • Hide your generally ugly server cabinet.

Cons:

  • The temperature could be very hot in summer.
  • Access to the basement can be difficult, and moving equipment to the basement may also difficult.
  • Can be far from your working desk, may require lots of walking.
  • Attic floor may not strong enough for holding heavy equipment.

Closet

Pros:

  • Easy to access because inside of your house or apartment, and providing a better work environment.
  • Can be quilt, depending on the material of the closet.
  • Blend in, hide your generally ugly server cabinet.
  • Reduce risk from accident damage and natural hazard such as flood, heatwave, bugs, etc… Also void damage from and to kids and pets.

Cons:

  • Serious heat problem due to closed environment.

You can have your equipment in different place of your home, for example, I have my network gear in a small network cabinet just above the front door and my main servers located in a larger cabinet in my study room.

Be in a fixed location not means you can not move them. You can have your server sets on a cabinet with wheels, or have a cluster of Raspberry Pi on a box you can move from one place to another. but your homelab equipment should not like your laptop or iPad that you can take where you goes to.

Use server cabinet and rack

Use server cabinet or rack can significantly make your homelab clean and make your device easily accessible because most of the industry equipment is design for rack mount.

But use or not use a server cabinet can depend on type and quantity of the equipment you have. For user with several heavy server and equipment, use standard server cabinet cloud be a better decision. The smaller size network rack is also a good choice for network device only user.

Get my new iPhone 11 Pro, not MAX

My iPhone X screen was broken when the world was put in lockdown mode two month ago due to the Coronavirus outbreak. Fortunately I was able to make backup to my computer and restore to my old iPhone 6s Plus, so I can make phone call and receive messages, but I need an new phone.

Few days ago, I order my new phone from Apple, and delivered this morning. It is an 256GB space gray iPhone Pro, with (PRODUCT RED) case.

iPhone 11 Pro may not be a choice for many people, the screen size and battery life is smaller and less than iPhone 11 Pro Max, but price was much higher than iPhone 11 without OLED. People intend to choose large screen phone when budget is not an issue, and for people want cheaper phone without sacrifice performance iPhone 11 will be their choice.

I want an better screen phone, therefor iPhone 11 with only LCD was out of my list. The new camera system on Pro series is also a plus.

My wife is using iPhone 11 Max Pro, although the screen is great, it too big and too heave for me to hold with one hand. With 5.8 inch iPhone 11 Pro I can still touch top left cornet with only my right hand, but there is no way to complete operate an 6.5 inch Pro Max without help from another hand.

As iPhone with 5G support will be available this year, I probably switch to new iPhone 12 and make this iPhone as the shortest-lived phone on my hands.

More: Infrared flood illuminator was turned on and captured by my camera.

Sublime Text 3 安装Package Control

Sublime Text 2 logo

Sublime Text可以算是我用过最棒的文版编辑器之一,不仅仅美观而且提供了强大的插件扩展功能,你几乎可以哟娜他来做任何想做的事情。使用Sublime Text 3的Beta已经有很长一点时间了,不过似乎开发团队仍然在添加一些功能以及修复Bug,而且看起来距离正式本版的发布还要有一段的时间。
由于Sublime Text的升级,Package control for Sublime 2不能直接在Sublime Text 3上正常工作,而安装Package control for Sublime 3 也不像在Sublime Text 2上那么的方便(就目前的状况看)。

Move to WordPress, and Why

wordpress on desk

I moved this site to official WordPress hosting (WP.com) about one year ago. I was using WordPress as my blog system since 2005 by moving away from Movable Type. My first site was setup on a LAMP server, then move to Media Temple Grid hosting, and then move to Linode VPS. After few problem and incident I decide move my site to managed WP with a paid plan.

Some times I receive message of asking comparing VPS hosted WordPress and WP.com, which is not an easy question to answer. I list reason why I move to WP.com and what I lost.

Why I move to WP.com

Security and Updates

Site security is important. And it was the main reason I move to a managed solution. I don’t mean how good the security in WordPress.com, but generally speaking they do security better than most VPS owner.

I work for a major IT company, and have client in Top 500 list. I know there will be risk of put my web server direct to internet. So when I deploy WordPress on my Linode VM, I put my Apache and database in an internal network behind a Haproxy.

Example of Haproxy with Let’s Encrypt

The software and proxy run on different VPS connect to private network with Firewalld on. I can only SSH to non-root account with ssh-key, not using password. And also have script for install Linux security update every 2 weeks.

Every thing looks great, but one day in 2017 I see my site was marked with security warning by Google. After check and file scan, several script has been found injected into WordPress plugin and program. The root cause was not found, the best guess was cause by security vulnerabilities in old WordPress version.

I have to complete rebuild those VM, and restore data from backup. But even update WordPress is not a difficult task, I did have experience of failed upgrade, and take me time to fix the problem. This is why I am looking for am managed service, and WP.com is the first option.

HTTPs and Let’s Encrypt ACME

I also using Haproxy for HTTPs offloading, and using Let’s Encrypt as my HTTPs certificate provider. But certificate provided by Let’s Encrypt have short expire date, so I setup auto renewal with ACME for update certificate every few weeks.

However, sometimes the auto renewal failed and causing Haproxy stop working. On the other hand WP.com helps me renew certificate without any problem.

Price

I currently using Premium Plan from WP.com costs me 8.00 USD per month. Compare to VPS plan, cost me 5.00+20.00=25.00 USD per month, save my 204.00 USD each year.

I also using premium theme included in my plan, so I do not need to spend extra cost to buy theme, or safe time for develop by my self. And I don’t need to spend time to maintain those VMs.

Cons

There are some limitation of using WP.com. Some of the limitation due to my 8.00 USD Premium Plan, and can be resolved by upgrade to more expensive plan. But other limitation can not work around easily.

Feature and Plugins

My Premium Plan don’t include plugin service. As a SaaS service, there is no more control other than WordPress dashboard panel, any extra feature not include in WordPress must require install a plugin. The cheapest plan with plugin support cost 25.00 USD per month.

Without plugin support some advance feature such as CDN, some show-box, google analytics are also limited.

Certificate

DNS and Domain Name

WP.com can providing one year free domain name for customer, but the best practice is register your domain name with major domain provider such as godaddy or name.com. I didn’t use free domain name provide by WP.com so I do not know how much I have to pay for next year, and I am not sure WP.com offer private Whois.

By default, WP.com require using their DNS, and WP.com DNS do support most of the DNS record type

But for user have large record list, or do not want to transfer DNS from HE.net or NS1.com, you can create a A record to WP.com ingress IP addresses. I using following IP address provided by WP.com support:

  • 192.0.78.24
  • 192.0.78.25

They can be different from your site, ask support for conformation.

SANS in HTTPs Certificate

Certificate issued by WP.com under common name *.wordpress.com. My domain name list as one DNS record in SANs, with other WP.com user’s domain name. I can not have a http certificate with only my domain name in SANs list.

China Problem

I living in China, and WP.com sometimes blocked in China. Currently I using a workaround for this issue. This only cause problem for people visit my site from China.

Log and Visiter Metadata

On WP.com, I can only see visitor information from Jetpack. If you have plugin support, other traffic analytics service such as Google can be support. But unlike VPS, I can access raw log from Haproxy or Apache web server can and send to ELK on my data platform for archive or analysis.

Summary

The only problem for me now is raw access log and no plugin support. I use CDN solve the certificate and access problem. I don’t know is any SaaS provider will provide those access log, but before I see better stable WP managed service, I will stay in WP.com, and perhaps buy the Business Plan.

Sublime Text 2,几近完美的文本编辑器

Sublime Text 2是一个非常不错的文本编辑器。其目的就是为Vi/Vim用户提供一款界面更加友好,功能更多的源代码编辑器。Sublime Text 2不仅仅支持语法加亮、基于正则表达式的查找与替换、自定义语法高亮、自动括号匹配、自动完成等功能,对Python的插件的支持也提供了更多“自定义”的空间。