If you are comfortable with PC hardware, don’t forget that after enough time (years) the thermal paste / thermal pad conducting the heat from the CPU to the heat sink may break down. If you are low on money and willing to do that much disassembly, that’s something to try if cleaning isn’t sufficient.
It’s also possible the fan is worn out and just needs replacement.
I had an issue with the fan of my NUC6i.
I just ordered a new fan on AliExpress for 4€. You need to see the reference wrote on the fan and then you could find easily a replacement part.
The problem with fanless chassis is they are difficult to find for older NUC.
And to finish, Don’t wait to replace the fan because in my case the nvme disk didn’t like that the fan didn’t work and it died too.