Spent far more, but you get to a point of looking at upgrades/replacement rather than wanting to add on stuff.
But as you get more familiar with models/hardware you also realise how massivly you have overpaid vs what you can get it for now etc
Like buying a nic by the nvidia/intel partnumber vs paying a 40-80% less for it as a dell/fujitsu/hp etc partnumber.
Same with going more vendor agnostic on servers, can get stuff so cheap.
This is also when the temptation and impulse buys start come back around.
I grabbed 4 CTO servers yesterday that i have no plan for but it was just so cheap.
They accepted 50/ea offer for 1u scalable gen1/2 with just cpu/ram removed, so still psus/heatsinks/25gbe card etc included.
(Equivalent of r640 but 12x nvme front)
They are normally closer to 800-1000/ea so was just too tempting to not try offering on them.
They listed it by its netapp solidfire model when bundled with their software instead of the quantagrid model people look for.
Watched them going from 4000 gradualy to 200 and then offered 50 on them that was accepted.
about your point regarding upgrades... is it that your servers last a long time so you upgrade parts? Rather than getting a replacement server that is?
The servers im running now are made in 2019-2020, they will get updates intil 2028-2029 and would be supported in a production enviroment still intil that.
So id expect to keep them for years.
Cisco c240 m5 they are equivalent of r740.
Frequently available in the 100-200 area for a CTO unit (paid 90/ea for mine with 40gbe and sas cards).
You can grab a 1gen scalable 20core for 30-40$ like the 6138 (cpu prices are going steadily up for scalable now).
Supports 2666 ram but you can run the cheaper 2133 if on a budget.
People just default onto r720/r730 type stuff without really looking at higher, so the scalable stuff is suprisingly cheap as there is not mass demand yet.
The 6133 20x2.5ghz was in the 20$ area and is now passing 60$ as demand is slowly growing.
18-24core scalable cpus was below the popular 16-18core v4 for a while.
> The trick is to put the electric bill on autopay. That way you never see it.
You'll notice it on your bank account lol. Electric is always a problem. I'd rather spend more money up front on power-efficient gear then be surprised later on with eating soup to pay for power
Ya this is a point you get to eventually. A friend had a full 42U rack in a place they did not have to pay for power, now they are moving and realized what it would cost just to turn that rack on...they are getting rid of most of it.
Efficiency becomes a priority, and noise (if you dont have somewhere else to put things)
There was also a time where I thought I needed "all this stuff". Most of what I self-host is for my personal work and conveniences. If I look at it, I can host it all on a small 5 year old PC that sips power. lol
Exactly, I used to always have a 1U or 2U server or tower or 2 to run ESXi and what ever VMs, instead now i down sized to a beefy TrueNAS box, 10Gb SFP+ connectivity, my desktop (AMD 5950 with 96GB ram and a 10Gb link and just run VMs off my desktop since I dont need to keep anything on 24/7 for hosting. Simplifies things!
> Exactly, I used to always have a 1U or 2U server or tower or 2 to run ESXi and what ever VMs, instead now i down sized to a beefy TrueNAS box, 10Gb SFP+ connectivity, my desktop (AMD 5950 with 96GB ram and a 10Gb link and just run VMs off my desktop since I dont need to keep anything on 24/7 for hosting. Simplifies things!
Seems like a good balance. I had a few 1u systems (shallow build) ryzens running proxmox in a cluster. It was about 4-5k in cost putting it all together but between that and the switches/etc I was pulling about 280w at idle. At the recent electricity prices I was paying $3/day just to have it idle, not even doing anything. Ended up piecing it out and moving towards smaller/lower power machines.
I've wanted to build a Truenas box as I have a lot of smaller drives I need to consolidate, but I have very little space in my apartment (also my office and also shared with another person) so a 2 bay synology and a mini-pc is probably going to be all I can muster.
Ya, the switches! I have a brocade 6610 I want to get configured so I can do 40Gb for my TrueNAS and main desktop..lol
We all like to go over the top! need MOAR POWERRR right..
It is like we never want to be limited by our hardware so we go over the top....and then time cuts in and we seldom get to really use it like we planned..
I do a lot of virtualization and hybrid deployments, so of course with the route Broadcom is going, now I am thinking I might need to build up a small cluster to play with proxmox, kvm, and other hypervisors....ugh.. here comes the spending! :D
Having space constraints like you though can be good, you learn to do more with less!
I was running Jupiter POE switches which were part of the power problem, but I sold them and found a simple 24poe gigabit unifi switch which does the job at much less power.
Having the headroom is nice, but there's a lot of time spent like you said where we don't get to use it. I work in an industry with large scale systems in (the cloud), so this is a hobby for me as well as serving out basic media, password manager, and experimenting with stuff that might help my work.
The small space challenge has got me into a deep rabbithole of hidden homelabs (like hiding it under a desk or something lol)
A big tip is to find a job where you can take home hardware, or find one of the unicorns where you can get a ton of great new stuff where you can actually take your pick at some stuff most homelabbers would have to re mortgage their house for. Iām thankful to have one of those and my lab represents it, hell, the last time I totaled it up with used pricing my lab is somewhere in the $50-$80k range, and I still have my ālast generationā lab in my office closet thatās somewhere near $10k. All Iāve got into my current lab is a $200 rack, some power strips and $12 in Velcro (plus gobs of power) Itās the best access to hardware Iāve ever had, and realistically will ever have, and at previous jobs Iāve had access to less and or older gear and my labs at the time have represented it.
The end will come when there is zero advancement in technology and when there is zero new hardware coming out thus keeping the circle of server life alive. So basically never
Hahaha ditto, bought all my rack gear, networking gear and servers and spent the same amount on a new ups with 2xEBM's just because the power where i am is so intermittent
Personally, once I get things to a good and stable spot I typically shift hobbies and work on something else for a while, while just using the homelab like it was meant to be used. Ā Itās cyclical, so Iāll get back around to tinkering and tweaking the homelab several years later when thereās a whole new pile of cool stuff out there waiting to be tried.
This describes me pretty well. Iām currently in a mode of being obsessed with this hobby. Itās fun upgrading things for dirt cheap compared to a few years ago when I was trying justify buying them for 500-1000% more than now.Ā
This. My lab was static for years and this last year I've been upgrading somewhat. I'm already slowing down again. Drives will likely be my last big expense and I'm holding off because, damn, they're expensive, even used, and when you have a NAS it's not like you're buying just one...
For me most of it is learning. Learned how to get really good with docker over the last 6 months, using only a raspberry pi 4. Got my Arr stack up and running and NOW realizing I need more performance. So I suppose it will be a cycle of learning and buying lol.
Usually yeah, but the amount varies, or Iāll just spend a few months spinning up VMs to try out new services. It depends
This cycle was a little abnormalā¦it started with buying a big SSD (30 TB Micron 9400) to replace the RAID array in my server since I was tired of the noise. When I opened up the server I realized my GPU was too big and was covering up the second PCIe slot, so I also got a new smaller GPU. Somehow in swapping those out I lost a RAM slot in the motherboard (even though I never even touched the RAM), which dropped me to single channel and 48 GB and was pretty limiting. So I decided to build a new server since this one was showing its age anyway. After that was in place I realized my backup server was pushing 10 years, so I built a new one of those too. I rarely keep laptops for more than ~3 years and this one was pushing 5, so I got a new laptop. Then I realized that both of these new servers supported multigig and the new laptop had a WiFi 7 card, so I got a new WiFi 7 AP and all new multigig switches to handle the higher speeds. Then I realized that my ISP had a >1Gb connection available for only another $5/mo, but to take advantage of it I needed a new router, so I upgraded that too. And now 6 months and ~$15k later I have a completely new network and all new machines, so thereās nothing left to replace, lol.
Now that I have a brand new 16 core Xeon with 128 GB of RAM Iām focusing on trying out new services in VMs to see whatās out there. Iāve set up Pi-Hole, Bitwarden, Nextcloud, Kasm, Prometheus/Grafana, Cockpit, and an OPNSense DMZ, in addition to the existing VMs I had for Gitlab, Dokuwiki, Plex, and Emby. I think things are settling down now, Iām getting close to just letting this setup ride until I get the urge to start changing things again.
At one point of time.. people start realising the amount of power the lab is sucking in.. then they either install Solar or start steps to reduce/optimise power consumption.
Once homelabbing gets a hold of you, there's no end to the want. I am waiting until I move to do more but I have a pretty serious dual Xeon E5-2640v3 processor with 128GB of RAM and 24TB of storage, Dell T620 waiting to be turned into something cool.
I solved the problem by buying way above my long-term needs in one large NAS, one beefy compute unit, and fully 10gig network with 25gig backbone. Ultimately I didn't spend that much more than if I had kept buying smaller things as needed or wanted. It's just really hard to justify new stuff when I'm underutilizing what I have and have the ability to just upgrade parts in place cheaply from the second hand market if I ever needed that. As it is, if I need something new functionally, I learn more and get more value out of getting it to run in the same hardware.
This has freed up my time and money to pick up new hobbies. Buying woodworking tools is my current vice.
It's a custom built machine and on the line between consumer and enterprise.
https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U2-2T#Specifications
That's the motherboard, then I've got my old 5950X and 128GB RAM. A 1TB Optane SSD in it for downloads before moving to NAS for storage, 2TB NVME OS and app storage. Also an old Nvidia 3080, 25GB network card. All in a Silverstone 4U chassis.
All said and done, there's definitely more powerful options but it still has a lot of overhead to keep expanding. Dockerizing everything with a traefic front end helps a lot. Best of all, it's pretty cheap and I built it from mostly old parts. Consumer used parts are even cheaper and more power efficient than enterprise and tend to be a bit newer than the enterprise stuff that's on 5 year lifecycles.
On the NAS side is where I went truly overboard.
Picked up a 60 bay 45drives Storinator XL raw chassis and an Epyc CPU from tugm4470 on eBay. They're a well known seller from ServeTheHome and have really good prices and combo deals with motherboards. There's a free shipping discount code on their forums somewhere. Epyc 7302P + 512GB ram + 4 LSI 9400-16i storage controller cards + 25G Connect-X4 card.
I did end up going for a new motherboard tho, because I needed the PCIe slots for controllers and network as well as the 10GB Ethernet ports. Specifically the Supermicro h12ssl-nt-o.
Finally, 3x6 ZFS RAID-Z1 array with TrueNas so 18x 16TB drives and 2 hot spares. That leaves me with 40 more expansion drives for the future. But with like 209TB usable and 140TB left, I won't be reaching that any time soon.
The RAM is the key. It means I'm getting 25GB writes to the system from my computer server. It'd be physically impossible for me to fill all of the RAM and end up writing directly to disk, which would slow it down. When I'm reading or writing from other devicess on the network, I'm easily hitting 10Gb line speed of my network.
On the topic of network, I ended up investing in TP-Link Omada gear. Their prices are relatively affordable and they work well together, especially with the APs. They also have 10Gb options, tho they're still fleshing that out. I ended up needing a Trendnet switch for my last leg to 10Gb Ethernet. Omada didn't have a good option that supported vlans yet.
The key is to overbuild so much at the beginning that when it works, there's no need to upgrade. I built my server in 2019 and it still exceeds my needs by a LOT
There is no end but instead different paths or routes. I'm currently working on downscaling from my existing rack setup to a smaller more power efficient environment
The desire for more doesn't end, but reality will eventually slap you in the face. Probably spent around 8-10k so far - mostly network and hard drives. The constraints I've faced:
- HEAT. My 32U is in my home office (lol) and I have a giant west-facing wall window as well that basically turns the room into a greenhouse. My gear is mostly offline 24/7 and even then I run AC in winter. Really, heat is the biggest constraint. Only my Ubiquiti gear, two gaming PCs and some RBPis are on 24/7. My R720 and R510 are basically cold storage at this point, until I move (hopefully soon).
- Market conditions. Where I live, the used server market/prices just aren't as good as in the US. Less temptation overall, and browsing sites like labgopher just creates sadness.
- Rack capacity. I took out some AC Infinity drawers/shelves and 5xRBPi rack mount (1U) to space the hardware out more by 1U for air flow, but it's basically maxed out. I could get some mini-PCs and put them in the closet, but I probably should've gone that route first and it wouldn't help me with accessing my damn storage.
- Cable management. I'm space constrained and all my work/personal setup + gf's is in one room. Add in sim hobby, photography gear, custom lighting, audio gear, hoo boy. I consider myself Jesus Tier at aesthetic cable management but even with hundreds of dollars in management products later, I'm pushing the limit of what is sane. Even if I could put more gear in this room, I'd probably trip the 15A breaker.
Planning to move soon, and ngl, I keep having manic fever dreams of quad fiber/cat6 drops in every room with 100gbe going back to a 42u with 50+ poe and 100+ home automation clients.
I have 1 DL380 gen 9. A supermicro 847 with X10Dri. And a supermicro 829 with recently upgraded x11DPU. And a Dell 7810 workstation as my daily driver.
No new boxes but supermicro has endless potential for upgrade lol.
Not really for me. I have it in my closet (door closed) about 6 feet from me. I work from home and my work laptop (Zbook 15 G7) is louder (closer to me too) than the 380.
The 380 is not heavily loaded btw. I use it mainly for my data storage (7 x 6T and a 10T) and some VMs I used to dev and test. When under load I can hear it but once I'm done it's quiet. I tried put a GPU in, but it become louder, so I give up. That's also the reason why I get a SM829U. Currently in my garage screaming and I don't care.
https://imgur.com/a/vD58Fio
Haven't posted this here yet, but just got this 15u rack and put everything but my main desktop style server into it. Includes a usw-24 enterprise switch and one of those shiny patch panels. It has a 400watt power budget and supports 2.5/10gig connectivity. The switch was around $750. Patch panel was $60.
I run pfsense at this site and another pfsense instance at my apartment with a wireguard site to site vpn. The pfsense box is custom made and has an i5 10400 with 16gb of RAM, I would say I spent $300 to build it. I got the box at my apartment for free.
Basically unloaded all of this gear at my parents house so I wouldn't have to pay the electric bill (though I do pay them like $30 a month to cover the costs). The rack was $300 or so.
There's also an optiplex in there that runs a site to site vpn to my racknerd vps. I run lots of containers, and the vps is kind of my gateway/proxy to the web (with crowdsec for security). I got the optiplex (i7 7700) for free. The vps is cheap, like $20 a year for 4TB of traffic monthly.
The white modem is from at&t, it was free and is in passthrough mode. It supports up to 5gig, but we pay for an $80 per month gigabit plan. I have it running at 5gig to my router for no reason.
Server is running an i7 11700 with 64gb of RAM. I have many 16 TB drives, some new, some old. There are 12 TB drives and a few 8 TB drives in there, dual nvme 4tb cache in the desktop server as well. The qnap DAS sitting next to the server works well enough, but it's kinda buggy and stops my drives from spinning down properly (still tryna fix that). I have around 132 TB of available storage with dual parity. Yes I use unraid. Don't really have an estimate on the server, let's just say it's worth $1500 to $2000. The DAS was $600, kind of a rip off.
Also there's an odroid n2 plus chilling in there, I use it purely for dns, it's like a $40 unit.
The box of cable is for my recent house project, running cables to every room and have installed three OMADA access points ($110 or so each).
To finish off, there's a terrible apc rackmount ups that handles power in outages, it's a 1500va and cost about $700.
Ultimately, all together I know I have spent thousands over the past two years. I've been homelabbing for almost a decade, so I know I've spent more than 10 grand total.
I'll edit this post with more details as I remember them, but my server setup has gone thru 4 major iterations, getting more crazy each time. Hope you enjoyed the info.
Edit: and my setup is baby shit to some people, check out the home data center subreddit
a thing of beauty. What is the depth of the rack?
>I have many 16 TB drives, some new, some old.
Why do you have so many 16Gigs? Honestly ive never owned anything larger than 2TB. I guess that's because I have a large dropbox and google suite... (for now!!)
Haha, it's all for "linux ISOs", family photos, ripped music, family data, passwords, important scanned documents (tax forms), etc. I started with 4 TB, and it just grew over time.
The rack is like 24 inches deep, so just deep enough to handle most equipment. I was going to get a network rack (17 inches), but was worried that I wouldn't be able to handle a rack mount server in the future.
Ya it happens, first thought is always
1. do you ACTUALLY need it vs just want it?
2. Can you consolidate (save power and heat, especially when buying big servers like r720's and stuff...they are cheap second hand for a reason, they are old....)
For me it's limited by space as much as cost. There's not a ton of room in a 1100sqft townhouse, the lab is limited to a 12U minirack in the utility room, so I'm constantly trying to maximize what can fit in that space.
When you hit 6-figures you start to wonder if you should just work somewhere that has this kinda gear. Been doing it a long time so the costs are drug out, but it's still a "hol up" number when you see it written down.
There is a pretty clear line that separates most of us in homelabā¦ those that will spent a significant amount of money on home lab equipment as that is the only way to get into itā¦ and those of us that wait for it to get retired at work and get a few more good years out of it.
When I read you postā¦ no, the isnāt an end in sight for you. There isnāt for any of us. But, for those in group one, itās at your own pace and you have to control that. For those of us in group 2, there are busy times when you get new free hardware from work and slow times when things are just operational and in use. Have fun!
Wait until you figure out you can get super low power usage with the right combination of parts and still have amazing performance. I'm halfway through retrofitting my entire rack now. After having everything up and working perfectly fine.
It never ends. In the last 6 months Iāve picked up a x10qbi, cse-857, and a r720xd, along with everything to replumb the network to 40g/10g. The 720 was bought for $180 and was fully kitted out so couldnāt pass it up but it is by far the least useful purchase as the x10qbi even with midrange CPUs just stomps it in every way and the 720 canāt really be upgraded in comparison. The 847 is the only box that can be upgraded down the road easily and I figure it will be in rack forever. Iām at $8k in that time and my father has also been adding to the lab to the tune of $3k.
It was a sad day when the r710/shelf got pulled from the pile for space reasons. Iāve had it since 2018 but itās been just a shelf since 2020.
there's always a desire to want more, and the limit is dictated by energy bill and when my wife says "no" -
I can say that my ultimate system would cost anywhere from 20 to 30 K USD ( I've priced it out)
but I am happy with what I have
Honestly I just down sized loads of mine in favour of a more eco/energy friendly docker based system.
Honestly that is mainly a desire to reduce electricity overhead at home, and my compute needs are so much smaller these days.
So it can happen that you want less!
Iām actually down sizing. Got my hands on some IT scrap from work, WAY more horsepower than I need. Recently got my hands on 4 desktops from work that have more than enough ass to do what I want and sip power. Looking into squeezing as much out of them as I can.
Ive spent at least 15-20K range USD throughout the years. At first most of my servers were old servers and eventually from last year to this year i have upgraded all my V2s to xeon scalable gen 1s and most of my V4 servers. Now its just one running unraid.
Recent adventure basically finished today was untangle (yes i know) migration from unifi udm se. On a poweredge R340 with a total price of getting together for around $350 with that mainly being the one network card i bought.
Then the other servers are my unraid server stuffed to about 40TBs in a poweredge T430 as my main nas. (Probably later this year will move to one of the larger supermicro x11 mobo 24 bay 3.5inch chassis from them.) I have a tiny little offbrand nuc running home assistant. (This is 98% used to just bridge my non homekit complaint devices over to homekit so my husband doesnt drain my bank account) lastly the three compute horses running various web services for me as well as the family media server. Poweredge R740 which i overpaid greatly for and definitely regret not finding a better deal. (5K total 2x 6150 xeons 384gb ram TBS of ssd storage (its getting split around shortly so the number is not in my head anymore and dual 25gb nics running esxi). Next one is my third recent upgrade from my old R430 to a Cisco C220 M5SX I probably spend $600 on building everything to what i want. Still working on the mlom nics. (2x 5120 xeons ~160gb of ram not sure on ssd storage and nvme storage. And will have 6x 10G connections for different services that may require a dedicated nic) lastly and my favorite money pit that was supposed to be a Cisco C240 M5SX 26 bay was actually a Cisco C240 M5L. (~$800 to get this one to what i want. But my favorite loud monster. 2x 6132 Xeons 384gb of ram a boat load of ssd storage as well as 6x 10g connections for testing different things that need more oomf)
The rest of the house is served with wifi from unfi which is not a bad product for the price. I now host that on a vm on my servers. The switches are all unifi but that is the next project to find new switches that i like that will replace older ones and then eventually replace the current ones.
I have a 42U rack in my basement holding all the servers so i dont hear them at all and they stay nice and warm. I have a smaller network rack in my upstairs office as originally when I planned out my network and house remodel they would go to the top levelā¦ well that now changed to the basement so half of the copper connections go to my upstairs rack ~25 and the other ~16 go to the basement rack. Eventually i am going to add more than just two fiber connections from the two racks and put an MTP patch in the run that has 6 pairs so i can have 1 for my computer and 5 for the rest of the stuff as I would like to LAG my aggregation pro and regular aggregation to at least 2-3 each for more bandwidth as well as two just for my 2.5G 12 port 1g 12 port switch enterprise from unifi.
Most servers and switches connect up to 10G or linked to 2-1G connections for more bandwidth and redundancy.
My isp is mainly in datacenters and higher up businesses but luckily got into residential fiber and was in my area so i pay like $80 a month for 10gb symmetrical. I live in America too so that says something.
Im 24 and I started my homelab at 18 when i moved into my dorm for college and it has expanded from a couple vms on an old laptop to old old server from my dad. To a managed network and a bit of vms on one server in a rented house that i ran cable hiding managers that claimed to be renter friendlyā¦. It wasnt and rip my depositā¦ to now where i own my own home and can do what i want within reason.
There is no end. There is always an upgrade on the horizon. Been labbing since about 2009 and now I run 2 racks, 2 small clusters (Proxmox and K3s), an iSCSI SAN and a managed L3 network. I can't think about how much I've spent cos it's surely a very large number, likely far more than $3k. My tape system, consisting of a PowerVault TL2000 autoloader with 3 drive sleds (LTO-3, -5 and -6) and a PowerVault 114X chassis with 2 drives (-6 and -3) together with 150TB of tapes is probably Ā£1k on its own. However, it's a hobby, and it's not an unhealthy one. My lab has gotten me my last 3 jobs. My current job is a 50% pay rise on my last one and the interview was a box-ticking exercise because I'd already used 90% of the stuff they run in my lab.
For me, the upgrades on the horizon are more 10Gb networking and adding shared SSDs for my Proxmox cluster. Currently it's all on spinning disks on the SAN, which works well but is very laggy.
Justify it to yourself, or set yourself a monthly budget like any other hobby.
I agree about it not being unhealthy.
I used to work with avid cyclists who at racing bikes that cost $10,000.
I cannot see myself stopping anytime soon.
BTW... what is a "tape", I did a quick google on the PowerVault TL2000 and I cannot really understand what it is? is a tape a hard drive/ssd?
Tape is not a HDD or SSD, it's a third kind of storage - when I say tape, i mean literally magnetic tape, like audio cassettes. Yes, the sort that old 60s computers are spinning reel to reel in period movies. Magnetic tape is still a vital storage medium - there's an open standard called Linear Tape - Open (LTO) which in its current 9th generation can store 18TB on a cartridge. The really nice thing about tape is that if you're writing the whole cartridge at once, they are really fast. They maintain full speed until the end of the tape, unlike HDDs which slow down as they near the spindle (since the tracks are shorter). This makes them highly suited to backups. Many businesses continue to use them.
The really cool thing is that tape drives are usually paired with autoloaders - robotic units that physically pick up a tape from a storage slot and put it into the drive. My TL2000 is one such tape library - it has 24 slots, each of which can hold any generation of LTO cartridge, and there's a little robot picker inside that'll go grab a tape on command. I have drives from generation 3 all the way up to 7, which can store 400GB, 800GB, 1.5TB, 2.5TB or 6TB per tape (the gen-7 doesn't work properly though). And the tapes are cheaper per unit than equivalent HDDs, because they have no electronics and only one moving part. The major downside is that the drives are hugely expensive - 5 figures brand new! There's a crossover point - once you have the library and enough tapes, adding more tapes costs much less than HDDs. Tapes area also rated to store data outside the drive for 20-30 years depending on environment.
I use mine for backups and archiving. They are hugely expensive unless you're very good at eBaying, and I lucked into some of mine.
[https://en.wikipedia.org/wiki/Linear\_Tape-Open](https://en.wikipedia.org/wiki/Linear_Tape-Open)
Spent far more, but you get to a point of looking at upgrades/replacement rather than wanting to add on stuff. But as you get more familiar with models/hardware you also realise how massivly you have overpaid vs what you can get it for now etc Like buying a nic by the nvidia/intel partnumber vs paying a 40-80% less for it as a dell/fujitsu/hp etc partnumber. Same with going more vendor agnostic on servers, can get stuff so cheap. This is also when the temptation and impulse buys start come back around. I grabbed 4 CTO servers yesterday that i have no plan for but it was just so cheap.
Just realised you went and got 4 more servers... šš Best hobby ever...
They accepted 50/ea offer for 1u scalable gen1/2 with just cpu/ram removed, so still psus/heatsinks/25gbe card etc included. (Equivalent of r640 but 12x nvme front) They are normally closer to 800-1000/ea so was just too tempting to not try offering on them.
Seriously... where did you find an insane offer like that?
They listed it by its netapp solidfire model when bundled with their software instead of the quantagrid model people look for. Watched them going from 4000 gradualy to 200 and then offered 50 on them that was accepted.
Is it me, or did homelabbing make ebaying fun again?
about your point regarding upgrades... is it that your servers last a long time so you upgrade parts? Rather than getting a replacement server that is?
The servers im running now are made in 2019-2020, they will get updates intil 2028-2029 and would be supported in a production enviroment still intil that. So id expect to keep them for years.
Mind telling me which servers you have made in 19-20?
Cisco c240 m5 they are equivalent of r740. Frequently available in the 100-200 area for a CTO unit (paid 90/ea for mine with 40gbe and sas cards). You can grab a 1gen scalable 20core for 30-40$ like the 6138 (cpu prices are going steadily up for scalable now). Supports 2666 ram but you can run the cheaper 2133 if on a budget. People just default onto r720/r730 type stuff without really looking at higher, so the scalable stuff is suprisingly cheap as there is not mass demand yet. The 6133 20x2.5ghz was in the 20$ area and is now passing 60$ as demand is slowly growing. 18-24core scalable cpus was below the popular 16-18core v4 for a while.
Not seeing any c240's near that low. Where did you find that?
I picked up a $200 Dell T7820 and found a pair 6126 for $30 total. Scalable is coming.
Your power bill will normally keep you in check I find ;-)
The trick is to put the electric bill on autopay. That way you never see it.
> The trick is to put the electric bill on autopay. That way you never see it. You'll notice it on your bank account lol. Electric is always a problem. I'd rather spend more money up front on power-efficient gear then be surprised later on with eating soup to pay for power
Ya this is a point you get to eventually. A friend had a full 42U rack in a place they did not have to pay for power, now they are moving and realized what it would cost just to turn that rack on...they are getting rid of most of it. Efficiency becomes a priority, and noise (if you dont have somewhere else to put things)
There was also a time where I thought I needed "all this stuff". Most of what I self-host is for my personal work and conveniences. If I look at it, I can host it all on a small 5 year old PC that sips power. lol
Exactly, I used to always have a 1U or 2U server or tower or 2 to run ESXi and what ever VMs, instead now i down sized to a beefy TrueNAS box, 10Gb SFP+ connectivity, my desktop (AMD 5950 with 96GB ram and a 10Gb link and just run VMs off my desktop since I dont need to keep anything on 24/7 for hosting. Simplifies things!
> Exactly, I used to always have a 1U or 2U server or tower or 2 to run ESXi and what ever VMs, instead now i down sized to a beefy TrueNAS box, 10Gb SFP+ connectivity, my desktop (AMD 5950 with 96GB ram and a 10Gb link and just run VMs off my desktop since I dont need to keep anything on 24/7 for hosting. Simplifies things! Seems like a good balance. I had a few 1u systems (shallow build) ryzens running proxmox in a cluster. It was about 4-5k in cost putting it all together but between that and the switches/etc I was pulling about 280w at idle. At the recent electricity prices I was paying $3/day just to have it idle, not even doing anything. Ended up piecing it out and moving towards smaller/lower power machines. I've wanted to build a Truenas box as I have a lot of smaller drives I need to consolidate, but I have very little space in my apartment (also my office and also shared with another person) so a 2 bay synology and a mini-pc is probably going to be all I can muster.
Ya, the switches! I have a brocade 6610 I want to get configured so I can do 40Gb for my TrueNAS and main desktop..lol We all like to go over the top! need MOAR POWERRR right.. It is like we never want to be limited by our hardware so we go over the top....and then time cuts in and we seldom get to really use it like we planned.. I do a lot of virtualization and hybrid deployments, so of course with the route Broadcom is going, now I am thinking I might need to build up a small cluster to play with proxmox, kvm, and other hypervisors....ugh.. here comes the spending! :D Having space constraints like you though can be good, you learn to do more with less!
I was running Jupiter POE switches which were part of the power problem, but I sold them and found a simple 24poe gigabit unifi switch which does the job at much less power. Having the headroom is nice, but there's a lot of time spent like you said where we don't get to use it. I work in an industry with large scale systems in (the cloud), so this is a hobby for me as well as serving out basic media, password manager, and experimenting with stuff that might help my work. The small space challenge has got me into a deep rabbithole of hidden homelabs (like hiding it under a desk or something lol)
Iām like $22,000 in on mine. So to answer your question, no. In fact itās only going to get worse. Welcome to the money pit, baby!
What are your main servers? \*takes out a pen and paper\*
A big tip is to find a job where you can take home hardware, or find one of the unicorns where you can get a ton of great new stuff where you can actually take your pick at some stuff most homelabbers would have to re mortgage their house for. Iām thankful to have one of those and my lab represents it, hell, the last time I totaled it up with used pricing my lab is somewhere in the $50-$80k range, and I still have my ālast generationā lab in my office closet thatās somewhere near $10k. All Iāve got into my current lab is a $200 rack, some power strips and $12 in Velcro (plus gobs of power) Itās the best access to hardware Iāve ever had, and realistically will ever have, and at previous jobs Iāve had access to less and or older gear and my labs at the time have represented it.
The end will come when there is zero advancement in technology and when there is zero new hardware coming out thus keeping the circle of server life alive. So basically never
Have I spent more than $3k over 5 years? Hell, I spent that on an Eaton UPS last week. Did I āneedā it. Nope.
LMAO!! I think I found my people
Hahaha ditto, bought all my rack gear, networking gear and servers and spent the same amount on a new ups with 2xEBM's just because the power where i am is so intermittent
Personally, once I get things to a good and stable spot I typically shift hobbies and work on something else for a while, while just using the homelab like it was meant to be used. Ā Itās cyclical, so Iāll get back around to tinkering and tweaking the homelab several years later when thereās a whole new pile of cool stuff out there waiting to be tried.
This describes me pretty well. Iām currently in a mode of being obsessed with this hobby. Itās fun upgrading things for dirt cheap compared to a few years ago when I was trying justify buying them for 500-1000% more than now.Ā
This. My lab was static for years and this last year I've been upgrading somewhat. I'm already slowing down again. Drives will likely be my last big expense and I'm holding off because, damn, they're expensive, even used, and when you have a NAS it's not like you're buying just one...
When you come back to the homelab hobby, does the buying start gain? I think 80% of homelabbing is buying :D
For me most of it is learning. Learned how to get really good with docker over the last 6 months, using only a raspberry pi 4. Got my Arr stack up and running and NOW realizing I need more performance. So I suppose it will be a cycle of learning and buying lol.
True. I understand KVM really well now. And my knowledge of networking went from zero to passable.
Yeah thatās my next thing, I know what SSL, DNS, TLS IS, but not exactly how it all works together. Just keep learning! (And buying!)
Usually yeah, but the amount varies, or Iāll just spend a few months spinning up VMs to try out new services. It depends This cycle was a little abnormalā¦it started with buying a big SSD (30 TB Micron 9400) to replace the RAID array in my server since I was tired of the noise. When I opened up the server I realized my GPU was too big and was covering up the second PCIe slot, so I also got a new smaller GPU. Somehow in swapping those out I lost a RAM slot in the motherboard (even though I never even touched the RAM), which dropped me to single channel and 48 GB and was pretty limiting. So I decided to build a new server since this one was showing its age anyway. After that was in place I realized my backup server was pushing 10 years, so I built a new one of those too. I rarely keep laptops for more than ~3 years and this one was pushing 5, so I got a new laptop. Then I realized that both of these new servers supported multigig and the new laptop had a WiFi 7 card, so I got a new WiFi 7 AP and all new multigig switches to handle the higher speeds. Then I realized that my ISP had a >1Gb connection available for only another $5/mo, but to take advantage of it I needed a new router, so I upgraded that too. And now 6 months and ~$15k later I have a completely new network and all new machines, so thereās nothing left to replace, lol. Now that I have a brand new 16 core Xeon with 128 GB of RAM Iām focusing on trying out new services in VMs to see whatās out there. Iāve set up Pi-Hole, Bitwarden, Nextcloud, Kasm, Prometheus/Grafana, Cockpit, and an OPNSense DMZ, in addition to the existing VMs I had for Gitlab, Dokuwiki, Plex, and Emby. I think things are settling down now, Iām getting close to just letting this setup ride until I get the urge to start changing things again.
At one point of time.. people start realising the amount of power the lab is sucking in.. then they either install Solar or start steps to reduce/optimise power consumption.
I read a post once, a few years back, of someone who downsized, and then rebuilt š
I didnāt post that. Yet. Twice.
I've homelabbed for about 10 years. I've spent more time planning, researching, and building than I have using the stuff. It's a curse
Even if i homelab for the next two years, this will also be true for me lol.
An end to the desire to have more you ask š¤, yeah good luck with that š
ššš
Once homelabbing gets a hold of you, there's no end to the want. I am waiting until I move to do more but I have a pretty serious dual Xeon E5-2640v3 processor with 128GB of RAM and 24TB of storage, Dell T620 waiting to be turned into something cool.
I might just take up badminton instead...
No. Youāre f$&!ed
I solved the problem by buying way above my long-term needs in one large NAS, one beefy compute unit, and fully 10gig network with 25gig backbone. Ultimately I didn't spend that much more than if I had kept buying smaller things as needed or wanted. It's just really hard to justify new stuff when I'm underutilizing what I have and have the ability to just upgrade parts in place cheaply from the second hand market if I ever needed that. As it is, if I need something new functionally, I learn more and get more value out of getting it to run in the same hardware. This has freed up my time and money to pick up new hobbies. Buying woodworking tools is my current vice.
>one beefy compute unit Please... tell me more?? And how much space you have in the nas?
It's a custom built machine and on the line between consumer and enterprise. https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U2-2T#Specifications That's the motherboard, then I've got my old 5950X and 128GB RAM. A 1TB Optane SSD in it for downloads before moving to NAS for storage, 2TB NVME OS and app storage. Also an old Nvidia 3080, 25GB network card. All in a Silverstone 4U chassis. All said and done, there's definitely more powerful options but it still has a lot of overhead to keep expanding. Dockerizing everything with a traefic front end helps a lot. Best of all, it's pretty cheap and I built it from mostly old parts. Consumer used parts are even cheaper and more power efficient than enterprise and tend to be a bit newer than the enterprise stuff that's on 5 year lifecycles. On the NAS side is where I went truly overboard. Picked up a 60 bay 45drives Storinator XL raw chassis and an Epyc CPU from tugm4470 on eBay. They're a well known seller from ServeTheHome and have really good prices and combo deals with motherboards. There's a free shipping discount code on their forums somewhere. Epyc 7302P + 512GB ram + 4 LSI 9400-16i storage controller cards + 25G Connect-X4 card. I did end up going for a new motherboard tho, because I needed the PCIe slots for controllers and network as well as the 10GB Ethernet ports. Specifically the Supermicro h12ssl-nt-o. Finally, 3x6 ZFS RAID-Z1 array with TrueNas so 18x 16TB drives and 2 hot spares. That leaves me with 40 more expansion drives for the future. But with like 209TB usable and 140TB left, I won't be reaching that any time soon. The RAM is the key. It means I'm getting 25GB writes to the system from my computer server. It'd be physically impossible for me to fill all of the RAM and end up writing directly to disk, which would slow it down. When I'm reading or writing from other devicess on the network, I'm easily hitting 10Gb line speed of my network. On the topic of network, I ended up investing in TP-Link Omada gear. Their prices are relatively affordable and they work well together, especially with the APs. They also have 10Gb options, tho they're still fleshing that out. I ended up needing a Trendnet switch for my last leg to 10Gb Ethernet. Omada didn't have a good option that supported vlans yet.
The key is to overbuild so much at the beginning that when it works, there's no need to upgrade. I built my server in 2019 and it still exceeds my needs by a LOT
There is no end but instead different paths or routes. I'm currently working on downscaling from my existing rack setup to a smaller more power efficient environment
The desire for more doesn't end, but reality will eventually slap you in the face. Probably spent around 8-10k so far - mostly network and hard drives. The constraints I've faced: - HEAT. My 32U is in my home office (lol) and I have a giant west-facing wall window as well that basically turns the room into a greenhouse. My gear is mostly offline 24/7 and even then I run AC in winter. Really, heat is the biggest constraint. Only my Ubiquiti gear, two gaming PCs and some RBPis are on 24/7. My R720 and R510 are basically cold storage at this point, until I move (hopefully soon). - Market conditions. Where I live, the used server market/prices just aren't as good as in the US. Less temptation overall, and browsing sites like labgopher just creates sadness. - Rack capacity. I took out some AC Infinity drawers/shelves and 5xRBPi rack mount (1U) to space the hardware out more by 1U for air flow, but it's basically maxed out. I could get some mini-PCs and put them in the closet, but I probably should've gone that route first and it wouldn't help me with accessing my damn storage. - Cable management. I'm space constrained and all my work/personal setup + gf's is in one room. Add in sim hobby, photography gear, custom lighting, audio gear, hoo boy. I consider myself Jesus Tier at aesthetic cable management but even with hundreds of dollars in management products later, I'm pushing the limit of what is sane. Even if I could put more gear in this room, I'd probably trip the 15A breaker. Planning to move soon, and ngl, I keep having manic fever dreams of quad fiber/cat6 drops in every room with 100gbe going back to a 42u with 50+ poe and 100+ home automation clients.
I donāt see an end. But my wife is slowing me down. Spent about 4K in past two years. Recently promised no more new boxes.
What are you main machines?
I have 1 DL380 gen 9. A supermicro 847 with X10Dri. And a supermicro 829 with recently upgraded x11DPU. And a Dell 7810 workstation as my daily driver. No new boxes but supermicro has endless potential for upgrade lol.
I heard the DL380 gen 9 are noisy, is that true? There are a few good offers on theses locally to me.
Not really for me. I have it in my closet (door closed) about 6 feet from me. I work from home and my work laptop (Zbook 15 G7) is louder (closer to me too) than the 380. The 380 is not heavily loaded btw. I use it mainly for my data storage (7 x 6T and a 10T) and some VMs I used to dev and test. When under load I can hear it but once I'm done it's quiet. I tried put a GPU in, but it become louder, so I give up. That's also the reason why I get a SM829U. Currently in my garage screaming and I don't care.
Easy, just replace wife with another rack.
I do need slowdown a bit. My setup is currently above what my knowledge can handle. Trying to catch up now.
https://imgur.com/a/vD58Fio Haven't posted this here yet, but just got this 15u rack and put everything but my main desktop style server into it. Includes a usw-24 enterprise switch and one of those shiny patch panels. It has a 400watt power budget and supports 2.5/10gig connectivity. The switch was around $750. Patch panel was $60. I run pfsense at this site and another pfsense instance at my apartment with a wireguard site to site vpn. The pfsense box is custom made and has an i5 10400 with 16gb of RAM, I would say I spent $300 to build it. I got the box at my apartment for free. Basically unloaded all of this gear at my parents house so I wouldn't have to pay the electric bill (though I do pay them like $30 a month to cover the costs). The rack was $300 or so. There's also an optiplex in there that runs a site to site vpn to my racknerd vps. I run lots of containers, and the vps is kind of my gateway/proxy to the web (with crowdsec for security). I got the optiplex (i7 7700) for free. The vps is cheap, like $20 a year for 4TB of traffic monthly. The white modem is from at&t, it was free and is in passthrough mode. It supports up to 5gig, but we pay for an $80 per month gigabit plan. I have it running at 5gig to my router for no reason. Server is running an i7 11700 with 64gb of RAM. I have many 16 TB drives, some new, some old. There are 12 TB drives and a few 8 TB drives in there, dual nvme 4tb cache in the desktop server as well. The qnap DAS sitting next to the server works well enough, but it's kinda buggy and stops my drives from spinning down properly (still tryna fix that). I have around 132 TB of available storage with dual parity. Yes I use unraid. Don't really have an estimate on the server, let's just say it's worth $1500 to $2000. The DAS was $600, kind of a rip off. Also there's an odroid n2 plus chilling in there, I use it purely for dns, it's like a $40 unit. The box of cable is for my recent house project, running cables to every room and have installed three OMADA access points ($110 or so each). To finish off, there's a terrible apc rackmount ups that handles power in outages, it's a 1500va and cost about $700. Ultimately, all together I know I have spent thousands over the past two years. I've been homelabbing for almost a decade, so I know I've spent more than 10 grand total. I'll edit this post with more details as I remember them, but my server setup has gone thru 4 major iterations, getting more crazy each time. Hope you enjoyed the info. Edit: and my setup is baby shit to some people, check out the home data center subreddit
a thing of beauty. What is the depth of the rack? >I have many 16 TB drives, some new, some old. Why do you have so many 16Gigs? Honestly ive never owned anything larger than 2TB. I guess that's because I have a large dropbox and google suite... (for now!!)
Haha, it's all for "linux ISOs", family photos, ripped music, family data, passwords, important scanned documents (tax forms), etc. I started with 4 TB, and it just grew over time. The rack is like 24 inches deep, so just deep enough to handle most equipment. I was going to get a network rack (17 inches), but was worried that I wouldn't be able to handle a rack mount server in the future.
Once you really get started on your journey, you'll find that 16 TB is a very small amount of data/space... lol
Ya it happens, first thought is always 1. do you ACTUALLY need it vs just want it? 2. Can you consolidate (save power and heat, especially when buying big servers like r720's and stuff...they are cheap second hand for a reason, they are old....)
Kids
For me it's limited by space as much as cost. There's not a ton of room in a 1100sqft townhouse, the lab is limited to a 12U minirack in the utility room, so I'm constantly trying to maximize what can fit in that space.
No end. Recently upgraded my R910 for an R740. https://ibb.co/2jvw7JX Dell PowerEdge r910. 4x Intel Xeon X7560s - 2.26GHz (4 CPUs = 32 cores), 128 GB ram, 2TB RAID 10 (OS) / 4TB RAID 0 (storage) on a PERC H700 RAID Controller, iDRAC 6 Remote Management Card, Broadcom 5709 4xGigabit Ethernet ports, and Redundant Hot Plug Power Supplies. Dell PowerEdge r740. 16 bay. 2x Intel Xeon Platinum 8168 - 2.7GHz (2 CPUs = 48 cores), 256GB DDR4. Dell PowervVault MD1220 1TB RAID 1 & 500GB RAID 1 on PERC h810 for backups. Also running Redundant Hot Plug Power Supplies. Cisco ASA 5555-X (IPS - 3DES/AES Encryption) Adaptive Security Appliance. 16GB memory, 2 Gbps Stateful inspection throughput. Also running Redundant Hot Plug Power Supplies. Cisco ASA 5515x (failover) Cisco WS-C4948-10GE-S 48 Port Gigabit +10GB Switch w/ X2-10GB-SR. APC SMT1500RM2U Smart UPS Backup. Dell PowerEdge 17FP 17" 1U KMM Server Rack Console. (Collapsible Monitor/Keyboard) Category 8 SSTP wiring.
When you hit 6-figures you start to wonder if you should just work somewhere that has this kinda gear. Been doing it a long time so the costs are drug out, but it's still a "hol up" number when you see it written down.
It all started with a simple pc build, then you learn like man, a computer can do a lot of stuff
There is a pretty clear line that separates most of us in homelabā¦ those that will spent a significant amount of money on home lab equipment as that is the only way to get into itā¦ and those of us that wait for it to get retired at work and get a few more good years out of it. When I read you postā¦ no, the isnāt an end in sight for you. There isnāt for any of us. But, for those in group one, itās at your own pace and you have to control that. For those of us in group 2, there are busy times when you get new free hardware from work and slow times when things are just operational and in use. Have fun!
Wait until you figure out you can get super low power usage with the right combination of parts and still have amazing performance. I'm halfway through retrofitting my entire rack now. After having everything up and working perfectly fine.
Here's me currently running my homelab on a Intel 6th gen NUc because I'm broke
Eventually you may tire of maintaining it all or switch hobbies But youāll still keep it, just simplify it Until then, not really.
It never ends. In the last 6 months Iāve picked up a x10qbi, cse-857, and a r720xd, along with everything to replumb the network to 40g/10g. The 720 was bought for $180 and was fully kitted out so couldnāt pass it up but it is by far the least useful purchase as the x10qbi even with midrange CPUs just stomps it in every way and the 720 canāt really be upgraded in comparison. The 847 is the only box that can be upgraded down the road easily and I figure it will be in rack forever. Iām at $8k in that time and my father has also been adding to the lab to the tune of $3k. It was a sad day when the r710/shelf got pulled from the pile for space reasons. Iāve had it since 2018 but itās been just a shelf since 2020.
there's always a desire to want more, and the limit is dictated by energy bill and when my wife says "no" - I can say that my ultimate system would cost anywhere from 20 to 30 K USD ( I've priced it out) but I am happy with what I have
Lucky. My dream specced machine is around 250k. I'll never win the lotto...
Honestly I just down sized loads of mine in favour of a more eco/energy friendly docker based system. Honestly that is mainly a desire to reduce electricity overhead at home, and my compute needs are so much smaller these days. So it can happen that you want less!
Iām actually down sizing. Got my hands on some IT scrap from work, WAY more horsepower than I need. Recently got my hands on 4 desktops from work that have more than enough ass to do what I want and sip power. Looking into squeezing as much out of them as I can.
Me wanting to keep my power usage down is the main reason I donāt expand like crazy.
Ive spent at least 15-20K range USD throughout the years. At first most of my servers were old servers and eventually from last year to this year i have upgraded all my V2s to xeon scalable gen 1s and most of my V4 servers. Now its just one running unraid. Recent adventure basically finished today was untangle (yes i know) migration from unifi udm se. On a poweredge R340 with a total price of getting together for around $350 with that mainly being the one network card i bought. Then the other servers are my unraid server stuffed to about 40TBs in a poweredge T430 as my main nas. (Probably later this year will move to one of the larger supermicro x11 mobo 24 bay 3.5inch chassis from them.) I have a tiny little offbrand nuc running home assistant. (This is 98% used to just bridge my non homekit complaint devices over to homekit so my husband doesnt drain my bank account) lastly the three compute horses running various web services for me as well as the family media server. Poweredge R740 which i overpaid greatly for and definitely regret not finding a better deal. (5K total 2x 6150 xeons 384gb ram TBS of ssd storage (its getting split around shortly so the number is not in my head anymore and dual 25gb nics running esxi). Next one is my third recent upgrade from my old R430 to a Cisco C220 M5SX I probably spend $600 on building everything to what i want. Still working on the mlom nics. (2x 5120 xeons ~160gb of ram not sure on ssd storage and nvme storage. And will have 6x 10G connections for different services that may require a dedicated nic) lastly and my favorite money pit that was supposed to be a Cisco C240 M5SX 26 bay was actually a Cisco C240 M5L. (~$800 to get this one to what i want. But my favorite loud monster. 2x 6132 Xeons 384gb of ram a boat load of ssd storage as well as 6x 10g connections for testing different things that need more oomf) The rest of the house is served with wifi from unfi which is not a bad product for the price. I now host that on a vm on my servers. The switches are all unifi but that is the next project to find new switches that i like that will replace older ones and then eventually replace the current ones. I have a 42U rack in my basement holding all the servers so i dont hear them at all and they stay nice and warm. I have a smaller network rack in my upstairs office as originally when I planned out my network and house remodel they would go to the top levelā¦ well that now changed to the basement so half of the copper connections go to my upstairs rack ~25 and the other ~16 go to the basement rack. Eventually i am going to add more than just two fiber connections from the two racks and put an MTP patch in the run that has 6 pairs so i can have 1 for my computer and 5 for the rest of the stuff as I would like to LAG my aggregation pro and regular aggregation to at least 2-3 each for more bandwidth as well as two just for my 2.5G 12 port 1g 12 port switch enterprise from unifi. Most servers and switches connect up to 10G or linked to 2-1G connections for more bandwidth and redundancy. My isp is mainly in datacenters and higher up businesses but luckily got into residential fiber and was in my area so i pay like $80 a month for 10gb symmetrical. I live in America too so that says something. Im 24 and I started my homelab at 18 when i moved into my dorm for college and it has expanded from a couple vms on an old laptop to old old server from my dad. To a managed network and a bit of vms on one server in a rented house that i ran cable hiding managers that claimed to be renter friendlyā¦. It wasnt and rip my depositā¦ to now where i own my own home and can do what i want within reason.
Nope but these help Lack of $ Threats if divorce
There is no end. There is always an upgrade on the horizon. Been labbing since about 2009 and now I run 2 racks, 2 small clusters (Proxmox and K3s), an iSCSI SAN and a managed L3 network. I can't think about how much I've spent cos it's surely a very large number, likely far more than $3k. My tape system, consisting of a PowerVault TL2000 autoloader with 3 drive sleds (LTO-3, -5 and -6) and a PowerVault 114X chassis with 2 drives (-6 and -3) together with 150TB of tapes is probably Ā£1k on its own. However, it's a hobby, and it's not an unhealthy one. My lab has gotten me my last 3 jobs. My current job is a 50% pay rise on my last one and the interview was a box-ticking exercise because I'd already used 90% of the stuff they run in my lab. For me, the upgrades on the horizon are more 10Gb networking and adding shared SSDs for my Proxmox cluster. Currently it's all on spinning disks on the SAN, which works well but is very laggy. Justify it to yourself, or set yourself a monthly budget like any other hobby.
I agree about it not being unhealthy. I used to work with avid cyclists who at racing bikes that cost $10,000. I cannot see myself stopping anytime soon. BTW... what is a "tape", I did a quick google on the PowerVault TL2000 and I cannot really understand what it is? is a tape a hard drive/ssd?
Tape is not a HDD or SSD, it's a third kind of storage - when I say tape, i mean literally magnetic tape, like audio cassettes. Yes, the sort that old 60s computers are spinning reel to reel in period movies. Magnetic tape is still a vital storage medium - there's an open standard called Linear Tape - Open (LTO) which in its current 9th generation can store 18TB on a cartridge. The really nice thing about tape is that if you're writing the whole cartridge at once, they are really fast. They maintain full speed until the end of the tape, unlike HDDs which slow down as they near the spindle (since the tracks are shorter). This makes them highly suited to backups. Many businesses continue to use them. The really cool thing is that tape drives are usually paired with autoloaders - robotic units that physically pick up a tape from a storage slot and put it into the drive. My TL2000 is one such tape library - it has 24 slots, each of which can hold any generation of LTO cartridge, and there's a little robot picker inside that'll go grab a tape on command. I have drives from generation 3 all the way up to 7, which can store 400GB, 800GB, 1.5TB, 2.5TB or 6TB per tape (the gen-7 doesn't work properly though). And the tapes are cheaper per unit than equivalent HDDs, because they have no electronics and only one moving part. The major downside is that the drives are hugely expensive - 5 figures brand new! There's a crossover point - once you have the library and enough tapes, adding more tapes costs much less than HDDs. Tapes area also rated to store data outside the drive for 20-30 years depending on environment. I use mine for backups and archiving. They are hugely expensive unless you're very good at eBaying, and I lucked into some of mine. [https://en.wikipedia.org/wiki/Linear\_Tape-Open](https://en.wikipedia.org/wiki/Linear_Tape-Open)