The FreeRTOS application which is included in the default image and mentioned there is located within the buildroot project for compiling images for Milk-V Duo boards, you can get it from GitHub.
In the appendix of the article linked above it is mentioned where you can find the FreeRTOS application sources within the buildroot project.
One more thing: you don’t need to reflash the whole image. In case of a TF, you only replace the fip.bin file on the boot partition and restart your Duo board with this new sd.bin (first core) + fip.bin (secondary) pair. (I suppose there might be a way of restarting the secondary core and the 80c51 core without any reboots, but I never tried it yet.)
Try reading Blink LED on FreeRTOS as well.