Come on, Alex!!! Three of the new Macs Mini together!!!
M
@meh2285
15
You should make more videos about this to compare how wifi vs eithernet vs thunderbolt connections impact performance, how larger models run, and other stuff that you are in the unique position to experiment with.
O
@op87867
8
10 base m4 Mac mini cluster here I come
S
@SirDealer
5
Please compare 4x Mac mini base model with 1x 4090 :D
K
@kumiho1729
4
I've been waiting for someone to do this test forever! I had a feeling you'd be the first. :D This is one of the main reasons I felt comfortable pulling the trigger on the base-model mini. I expect to use it in some fashion even a decade (or two) from now.
F
@ftab
3
Great! Instead of buying 16 GPUs and really large expensive boards, power supplies, cases, I can just buy 3 MBP M4 Max 128 gigglebytes and cluster them together for only $15,000!
N
@nirglazer5962
2
how does this actually work? you're not actually sharing the compute power right? basically it determines to which computer to send the query to, and then that computer shares the result with the one you're working on? would combining 3 of the same computer be beneficial or just repetitive?
_
@_hmh
2
This is impressive. If this was for a real-world use case, I’d implement these optimizations:
- Don’t use the NAS since it introduces a single point of failure and it is much slower than directly attached storage. For best performance, the internal SSDs are your best choice. Storing the model on each computer is ok. This is called “shared nothing”
- Use identical computers. My hypothesis is that slower computers slow down the whole cluster. You would need to measure it with the Activity Monitor
- Measure the network traffic. Use a network switch (or better two together with ethernet bonding for redundancy and speed increase) so that you can add an arbitrary number of computers to your setup
- Measure how well your model scales out. If you have three computers and add a fourth, you would expect to get one third more tokens per second. The increase that you actually get in relation to the computing power you added, defines your scale out efficiency.
- use identical computers to get comparable results
- Now you have a perfect cluster where you can remove any component without breaking the complete cluster. Whichever component you remove, the rest would still function.
D
@DS-pk4eh
2
This is a nice POC. Another great video from Alex.
I would definitely prefer having 10Gb switch and having everything connected to it (there are some 8 ports for 300USD). More stable to actually work, and probably, less messy. Maybe getting miniPC with 10Gb port and some decent amount of memory? Its shame Apple has such a big tax on memory and storage upgrade.
There is also Asustor 10Gb SSD only NAS device with 12x SSD slots (Flashstor 12 Pro).
S
@sreeharisreelal
2
I have a question regarding the installation of SQL Server Management Studio (SSMS) on a Mac. Specifically, I would like to know if it is feasible to install SSMS within a Windows 11 virtual machine using Parallels Desktop, and whether I would be able to connect this installation to an SQL Server that is running on the host macOS. Are there any specific configurations or steps I should be aware of to ensure a successful connection between SSMS and the SQL Server on macOS? Thank you!
M
@meh2285
15
You should make more videos about this to compare how wifi vs eithernet vs thunderbolt connections impact performance, how larger models run, and other stuff that you are in the unique position to experiment with.
O
@op87867
8
10 base m4 Mac mini cluster here I come
S
@SirDealer
5
Please compare 4x Mac mini base model with 1x 4090 :D
K
@kumiho1729
4
I've been waiting for someone to do this test forever! I had a feeling you'd be the first. :D This is one of the main reasons I felt comfortable pulling the trigger on the base-model mini. I expect to use it in some fashion even a decade (or two) from now.
F
@ftab
3
Great! Instead of buying 16 GPUs and really large expensive boards, power supplies, cases, I can just buy 3 MBP M4 Max 128 gigglebytes and cluster them together for only $15,000!
N
@nirglazer5962
2
how does this actually work? you're not actually sharing the compute power right? basically it determines to which computer to send the query to, and then that computer shares the result with the one you're working on? would combining 3 of the same computer be beneficial or just repetitive?
_
@_hmh
2
This is impressive. If this was for a real-world use case, I’d implement these optimizations:
- Don’t use the NAS since it introduces a single point of failure and it is much slower than directly attached storage. For best performance, the internal SSDs are your best choice. Storing the model on each computer is ok. This is called “shared nothing”
- Use identical computers. My hypothesis is that slower computers slow down the whole cluster. You would need to measure it with the Activity Monitor
- Measure the network traffic. Use a network switch (or better two together with ethernet bonding for redundancy and speed increase) so that you can add an arbitrary number of computers to your setup
- Measure how well your model scales out. If you have three computers and add a fourth, you would expect to get one third more tokens per second. The increase that you actually get in relation to the computing power you added, defines your scale out efficiency.
- use identical computers to get comparable results
- Now you have a perfect cluster where you can remove any component without breaking the complete cluster. Whichever component you remove, the rest would still function.
D
@DS-pk4eh
2
This is a nice POC. Another great video from Alex.
I would definitely prefer having 10Gb switch and having everything connected to it (there are some 8 ports for 300USD). More stable to actually work, and probably, less messy. Maybe getting miniPC with 10Gb port and some decent amount of memory? Its shame Apple has such a big tax on memory and storage upgrade.
There is also Asustor 10Gb SSD only NAS device with 12x SSD slots (Flashstor 12 Pro).
S
@sreeharisreelal
2
I have a question regarding the installation of SQL Server Management Studio (SSMS) on a Mac. Specifically, I would like to know if it is feasible to install SSMS within a Windows 11 virtual machine using Parallels Desktop, and whether I would be able to connect this installation to an SQL Server that is running on the host macOS. Are there any specific configurations or steps I should be aware of to ensure a successful connection between SSMS and the SQL Server on macOS? Thank you!
This is one of the main reasons I felt comfortable pulling the trigger on the base-model mini. I expect to use it in some fashion even a decade (or two) from now.
- Don’t use the NAS since it introduces a single point of failure and it is much slower than directly attached storage. For best performance, the internal SSDs are your best choice. Storing the model on each computer is ok. This is called “shared nothing”
- Use identical computers. My hypothesis is that slower computers slow down the whole cluster. You would need to measure it with the Activity Monitor
- Measure the network traffic. Use a network switch (or better two together with ethernet bonding for redundancy and speed increase) so that you can add an arbitrary number of computers to your setup
- Measure how well your model scales out. If you have three computers and add a fourth, you would expect to get one third more tokens per second. The increase that you actually get in relation to the computing power you added, defines your scale out efficiency.
- use identical computers to get comparable results
- Now you have a perfect cluster where you can remove any component without breaking the complete cluster. Whichever component you remove, the rest would still function.
I would definitely prefer having 10Gb switch and having everything connected to it (there are some 8 ports for 300USD). More stable to actually work, and probably, less messy.
Maybe getting miniPC with 10Gb port and some decent amount of memory? Its shame Apple has such a big tax on memory and storage upgrade.
There is also Asustor 10Gb SSD only NAS device with 12x SSD slots (Flashstor 12 Pro).
This is one of the main reasons I felt comfortable pulling the trigger on the base-model mini. I expect to use it in some fashion even a decade (or two) from now.
- Don’t use the NAS since it introduces a single point of failure and it is much slower than directly attached storage. For best performance, the internal SSDs are your best choice. Storing the model on each computer is ok. This is called “shared nothing”
- Use identical computers. My hypothesis is that slower computers slow down the whole cluster. You would need to measure it with the Activity Monitor
- Measure the network traffic. Use a network switch (or better two together with ethernet bonding for redundancy and speed increase) so that you can add an arbitrary number of computers to your setup
- Measure how well your model scales out. If you have three computers and add a fourth, you would expect to get one third more tokens per second. The increase that you actually get in relation to the computing power you added, defines your scale out efficiency.
- use identical computers to get comparable results
- Now you have a perfect cluster where you can remove any component without breaking the complete cluster. Whichever component you remove, the rest would still function.
I would definitely prefer having 10Gb switch and having everything connected to it (there are some 8 ports for 300USD). More stable to actually work, and probably, less messy.
Maybe getting miniPC with 10Gb port and some decent amount of memory? Its shame Apple has such a big tax on memory and storage upgrade.
There is also Asustor 10Gb SSD only NAS device with 12x SSD slots (Flashstor 12 Pro).