There is an inherent risk of system instability when it comes to mixing ram even if it's the same speed/CAS latency e.g XMP not working properly, losing dual-channel, BSOD but sometimes it can work just fine, very hard to tell unless you have actually tried it for a long period of time.
If you have time spare to do any potential troubleshooting & testing of memory (e.g using OCCT/TM5) then you can try mixing them together. If you are really busy and don't want to deal with any potential system instability then go for the safer route.