I would recommend having a read through the documentation here, specifically section 2.2:
It shows you how to customize styling by editing scss files.
If you want a more hands-on approach to learning how to use styling in Mendix, I'd recommend this learning path:
For borders specifically, you can simply add something like this into the right place in your scss files:
border: 5px solid red;
You can inspect that radio button and add the code {border: 5px solid red;} in element.style section if it working then You can make changes in the file that contain radio button code