Id expect them to be as I see nothing indicating any sort of new ‘coupons’.
Their last post was merely updated SoC specs, however Im betting that its NOT going to be out this year if they were still changing specs… or perhaps they had additional hw in there that worked better than expected on bring up… which for end of 2024 I would expect the bring up to be well under way by now… who knows there is so little information…
Happy Lunar New Year to those who celebrate…
[EDIT]
Still looking forward to getting my hands on one of these if the price doesn’t end up being ridiculously high, for mostly untried arch/ISA… i.e. they’re going to do some painful ‘buying’ into the market outside of niche embedded use…
I had just bought a RockPro64 to build a NAS… This will replace it, most certainly. If I can have a NAS and also run an LLM in my own network, this’ll absolutely rule.
What is the expected power consumption/PSU requirements on this?
And will you be working to upstream the SoC and board into mainline Linux?
I have been experimenting a lot with a VisionFive2 - but it’s 4 cores are hardly enough to really designate it as a build server for anything. With this amount of cores and performance, the situation is wholly different =)
Signed up just to earmark this forum for future news. Actually ultra exciting.
Side note: The LLaMa 65B above might be un-quantisized; meaning, there is room to grow, which would actually be nuts. o.o
Is there any further update on the SG2380 cpu development / production ? last update was Dec. It would be nice for a monthly or bimonthly update on both the cpu and Oasis board development! Is there a different link I would be following ?
Spec question here, this is fromt he internet, multiple sites:
23 October 2023 - Milk-V Oasis is a mini ITX board with a 16-core RISC-V processor and 20 TOPS NPU
From the specs image
Supports 16TOPS(INT8 MatMul)
From Sifive page:
The SiFive Intelligence Extensions enable the X280
processor to achieve a best-in-class 2.25 TFLOPs
(bfloat16 MatMul) or 4.6 TOPS (INT8 MatMul) - this is in a quadcore(at 2.13GHz, on an 8 core configuration we are looking at best 9TOPS (INT8 MatMul)(unless you have different specs for the core)
So realistically how many real TOPS(int8) will the oasis board support for AI?
20?
16?
9?
What is the clocking on the x280?
“tape out”? I guess this is a term for “ready to produce”?
Either way, I am very much looking forward to this chip! Even picked out a nice case for it.
tapeout is a silicon design term
In the past programming and computing was mechanical so you had perforated tapes, them it was magnetic tapes
These days tape out is just a term, its before the chip goes to production
And also with chips you got revisions, so when the new ones come, they will go to boards, the boards will be tested, so will software.
Then you get chip revisions and sub revisions like an A3 (three redesigns) or D1(how many per major revision + the 1)
When I worked for Broadcom we saw D2’s for example, that’s silicon bugs in multiple revisions
Thanks @bruce_loco !
I did know of punchcards, tapes and the like; one of my friends works at a company that utilizes tape drives for periodic backups - and by chance, I have such a tape - but no drive - in my closet… no idea how or when that happened exactly, though. o.o
So basically, what Wei ment, is that the chip currently has not seen any physical revision (-> still in design phase) and thus no details are set “in stone” yet … correct?
yeah, that’s pretty much it
Also it’s specs are not guaranteed, as the x280 might change?
They might as well incorporate the Hailo-8 26tops chip for AI, that would have a kick to it
I read the specs for the Hailo-8 chip … I need to up my AI-game in terms of understanding hardware requirements. Right now, I just use a 4090 with oobabooga - long term, it’d be awesome to have such a mini-itx box sitting somewhere in my home as part of my homelab to serve an LLM on a great power budget, which the Oasis seems to offer. So honestly, I just hope it can be that
Got any good pointers for where/how I could speedrun hardware requirements for LLMs in more detail, beyond memory usage? For that, I will just hope that quantization will be useful; but in terms of how many TOPS equate to what “expectable” performance, I know nothing - and it’ll become more important the closer the Oasis comes to release; would love to understand more of what I might pre-order (haven’t taken the coupon just yet.)
Honestly I am at the beginning stages of this game. I do know that having optimised machines for AI is the way to go, but also have lots of memory.
I thought of buying 4 Hailo-8’s and stuffing them on a Hyper quad(for ssds)
I am just checking whether the M.2 hyper quads can take the non NVME cards.
If it works, I can buy one Hailo 8 M.2, see how it works, then add another one - these are 200$ a pop.
But having 20 tops on the Oasis and having the ability to add extra AI cards that are low power, now you are talking speed and low power consumption.
But we would need the high memory version of the Oasis - the possible 96GB card
The Hugging face forum has a lot of info on machine learning, model training, etc
I am a newb like you
look at this, 4 H8, but its around 1000 euros, plus its MXM, so an MXM to PCIE adapter would be required…
I see the potential opportunity for good news from that bad news, in that it might allow for a slightly later patch revision of the SiFive IP.
I guess it would be pointless in asking for an update to the timeline, because some Mini-ITX PCB (Printed Circuit Board) design work could still happen in parallel even without the SG2380 being fully taped out.
EDIT: I just read this post (from October 2023, so the dates given are probably not perfect) http://forum.sophgo.com/t/about-the-sg2380-oasis-category/ I’m wondering if the “RTL Freeze” has happened already (The step before the tape out).
For a data server you could easily connect up to 10 SATA 3.0 drives. There are 4 SATA ports and in theory you could use the “M.2 M Key for NVMe SSD(PCIe 3.0 x4 ~3.938 GB/s)” slot and connect a “M.2 NVME to SATA 3.0 (~600MB/s) Card 6 Ports” (usually ASM1166 chipset - PCIe Gen3 x2 ~1.969 GB/s).
My question is if there will be any Oasis boards available with ECC RAM ? It is not essential for normal use, but if you planned on having 40 to 320 TB of disks attached it would be extremely preferable.
