True, but it not like intel is doing nothing, e.g., new NUCs are designed for Core Ultra 9 processors. You do not necessarily need some AI co-processor to do any AI, a CPU also works. And you can look up the thermal design power of the Core Ultra 9 226V or 288V, they come with integrated NPU: 17 to 30 W @40/48 TOPS for the NPU. Plus, you have an additional GPU with 53 to 67 TOPS. I am not saying that the Oasis will be in any way bad, but let’s keep everything in perspective.
1 Like
I feel it is also important to point out, that especially LLMs have a rather high memory cost. Let’s take for example the RTX 3060. I’ve read it has about 100 TOPS INT8. This is considerably more than any of the parties involved in getting the Oasis to market are projecting for the Oasis. The RTX 3060, however, only has 12 GB of VRAM. Even comparatively small LLMs smash that. So in practice, there are models on which (even without speculating on the Oasis’ performance) the Oasis will outperform a 3060 due to the 3060s rather small VRAM.
2 Likes
This is a valid point, no doubt. But, as mentioned above, you can also compare the performance of the Oasis to a bog-standard intel NUC with an intel Core Ultra 9 226V. When the predictions of the manufacturers come true regarding the performance, the intel system will win the comparison even for LLMs hands down. My point is: The Oasis could be a game changer, but not because it outperforms other systems. I see it as a demonstrator that RISC-V can be used in a versatile desktop machine that can do a lot of things (including AI, but not limited to AI) at a reasonable price point. But once the AI fans try to influence the design decisions like including expensive LPCAMM2 memory, the whole nice idea might fail. You will end up with an expensive system that has lower CPU performance, lower AI performance and an unstable software ecosystem. It does not compare well to other systems with a similar price point, like, for example, a normal intel NUC. The complete Oasis endevour becomes meaningless. RISC-V systems will look inferior to what intel offers. Not a good techonolgy demonstrator for RISC-V. So, I it would be nice if all participants could keep their expectations reasonable and know beforehand what they will get.
3 Likes
Ah sorry I didn’t see that. Was not really trying to argue that the Oasis will be a game changer for AI performance. Just wanted to point out that comparing an NPU that uses system memory with a GPU is not really that apples to apples.
I second that. Me personally, I don’t care a bit about AI. My interest in the Oasis comes from wanting RISC V to succeed as a daily driver ecosystem. I see a lot of push to high power RISC V as coming from the AI bubble (Tenstorrent for example). While I welcome the support, I fear that RISC V will be absorbed into the AI bubble and pop with it, similarly to how the Crypto/Web3 hype absorbed quite a lot of actually promising technologies only to take them down with them on the way down.
8 Likes
Yes, A lot of the recent developments within risc-v is just a byproduct of the AI craze. Sophgo is currently just focused on creating TPU’s because there is a large demand for them. Visiting their website makes that pretty obvious.
I also think it’s clear that the SOCs for the Duo chips are developed with IP camera’s in mind, judging by the datasheets and even the demos.
So really the AI tech is what is enabling boards like the Oasis to even exist, even though Milk-V uses them for a different usecase. You’re not special to be here with a desktop experience over an AI usecase in mind. The promise of a desktop experience is literally in the product description of the oasis and in the title of this post.
Still have you seen the SG2380 product site? The chip is literally developed for and targeted towards AI usecases.
https://sg2380.org/
Who knows if it even is a bubble, that depends on if the TPU’s actually prove useful or not. In any case I don’t see a reason why Risc-V will die with it. Are you suggesting all chinese vendors will just move to another architecture just after perfecting Risc-V chip development because demand for TPU’s dropped? I think their seeming dependence on risc-v shows that it will stick around, and that dependency can only grow larger.
Personally I think it’s the opposite. People want the LPCAMM2 because they want to see a true RISC-V desktop experience.
Could be, but I am not sure that all participants in this forum are aware that there are other factors that influence the desktop experience a lot more than the amount of RAM. Each to their own. In my eyes, a higher price point for the complete system will make the product much less appealing. And I do not see the Oasis as a long-term investment where I might upgrade the RAM configuration later. Also, without those over-hyped AI promises, I see no application that would benefit from the huge amount of memory some people want to install with LPCAMM2 modules. The CPU performance is nothing to call home about, compared to the intel or ARM world. When the system gets more expensive, the price-performance ratio is disadvantageous, the entry barrier higher, it could prevent the adoption of RISC-V for the desktop. You simply attract more developers if they do not have to invest considerable amounts of money in a system before being able to use it. I, for sure, would consider it a waste of resources to install an LPCAMM2 with an expected cost of more than 200 USD in such a system, both financially and technically.
1 Like
Regarding RAM, I guess a hybrid solution is not even considered.
I mean, having some RAM soldered but also (working) sockets to add LPCAMM modules.
This way it would have been possible to ship a cheaper system, but allow users to upgrade assuming the cost.
The problem is the SG2380 256-bit memory bus width and if they shipped with only a 128-bit memory module installed (soldered to the board), that would require some serious boot code tweaking to get the board to boot with half the performance. And generally in extremely high speed circuitry you need impedance matched transmissions lines that are all the exact same physical length, and that becomes particularly complex with two different types of memory. And then you will have people who expect two different sizes of memory from two different vendors to work, which would add even more complexity (which would probably be solved by degrading performance more). Would you keep things simple and access all memory in degraded 128-bit mode (complex solutions add new problems), or would you access 128-bits of the smaller sized RAM and 128-bits of the larger size in parallel for maximum throughput and then access the remainder of the larger memory modules in degraded 128-bit mode. This would give uneven performance depending on where application instructions and data are in RAM. I suspect that a hybrid solution would just add many new and complex bugs.
3 Likes
I think I would personally just put a single 32GB dimm in, with crucial that should come in at 200 euro’s. I am not sure what the price of the same amount of LPDDR5 would be.
In any case there is currently only a single vendor for LPCAMM2 (crucial), that’s subject to change. By the time the Oasis actually ships there might be more options.
If not then I think a serious concern is availability/accessibility of the modules in the first place. I don’t think Crucial does world-wide shipping.
So I am still on the fence, I would love the Oasis to ship with an overall lower cost. But I would also love being one of the first to play with LPCAMM2 modules. To me it depends on the price of LPDDR5 and the future price of LPCAMM2, which are unknowns.
Simply buy a Thinkpad P1 Gen 7 (for around 3000 USD), then you can try LPCAMM2 modules already today. It might be fun for 5 seconds to plug in a 644 pin module, but for me, the play value would be over pretty soon. In day to day use, I see no difference, as the chips on the LPCAMM2 module will be LPDDR5X anyway, the computer will feel totally the same with soldered RAM. Unless the LPCAMM2 connector on the Oasis would be unreliable. On the positive side, Samsung LPCAMM2 modules will also be available soon. But in the beginning, you would still need to pay high-end prices to get your memory for your low-end desktop. Sorry, still makes absolutely no sense for me.
Not everyone requiring the larger amount of memory will be using it exclusively for AI and such. Rocky Linux mentioned above about using Oasis as the basis for their RISC-V deployment for example. An important task then will be to set up a few systems for an automated build farm. The extra memory along with the built-in SATA ports which allows easily RAID-ing together four cheap 20TB drives will come in handy in order to point the systems to the repo and start crunching through all the thousands of source packages continuously to get this going. There is also regression testing of resulting binaries, etc. This can’t realistically be done on a 4GB VisionFive. Or even a DC Roma laptop.
In addition at this early stage all that memory will help Rocky Linux developers tremendously to overcome the immaturity of the software stack or compensate for worthless hardware like the embedded Imagination GPU. As even noted, many parts of Linux like the kernel, gcc, low level code has not been optimized for the new architecture yet hence the cause of the low performance seen in some real-world experience and benchmarks now that physical boards have released into the wild recently. So on the desktop side completing such tasks as compiling the kernel over and over again, testing the various GUI environments and apps, being able to run multiple instances of VM to try out different test kernels being worked on will be a lot less painful with more memory and cores. Contrast that to trying to compile the kernel on a smaller SBC for just one run can takes hours or days.
Unfortunately throwing hardware at the problem is an evil necessity at the moment, on the road to getting a smooth and performant Linux system sooner rather than excruciating later. Oasis is the first board bearing enough useful hardware feature-set at a reasonable price to make it viable for more non-professionals and tinkerers and not-for-profit groups without deep pockets to get a play at RISC-V, and hopefully advance it further. The more contributors the merrier as they say.
5 Likes
I honestly hope that they have an automated build system that allows to cross-compile for different architectures, similar to the old CentOS one. Automated testing would be more of use-case for real hardware.
I honestly hope that they have an automated build system that allows to cross-compile for different architectures
Cross compiling is very slow, and not even possible for many applications
2 Likes
Is it? Why do you think so? Usually, the opposite is true for slower platforms.
3 Likes
Why do you think so?
Because I used to do it for ARM from x86, and it’s slow. Even the 4 core Neoverse N1 VPS that I have is faster than my 32 core Xeon server.
2 Likes
I don’t think it’s likely that RISC V would die just because the AI hype is over. It just leaves an awful taste in my mouth that high performance RISC V is so strongly connected with AI right now.
1 Like
It appears I’m one of the few individuals on this platform interested in utilizing this as an AI machine.
The memory bandwidth, NPU, and upgradability of memory make for a compelling low-power, low-cost offering compared to GPU or Apple M series costs.
For instance, the Gemma 2 27b should be capable of running quantized Q6 at approximately 7 T/s, which is more than sufficient for spoken text processing in an LLM assistant.
Provided, of course, that software support doesn’t take the approach of ‘it can run LLaMA2, and that’s good enough.’ I would much rather have LLaMA.cpp integration.
1 Like
So it looks like a majority of the community want the Oasis to go with LPCAMM2, which I’m kinda bummed about, availability is going to be tricky.
@hoka or @WeiMouMou if it goes the LPCAMM2 route, would there be potential to offer a bundle pack at a higher price point to include some LPCAMM2 with the board?
It’s going to be really hard to get any in my region so being able to buy some with the board would be ideal, if not I might not be able to get the board at all unfortunately
3 Likes
@WeiMouMou I’d be interested to hear about progress on this. How far has it come along. Has the design been finalized? Tapeout? Do you have it on hand?
I’m guessing we might not meet the originally planned October 2024 release timeline. If so, any estimates of what the new expected release date will be?
6 Likes
Seconding this, I would love an update, just generally how things are going and what not, especially interested in what decision was made on RAM LPDDR5 vs LPCAMM2
1 Like