You can load the certificates into the recources folder as a .pfx and load this in the java code:
File f = new File(Core.getConfiguration().getResourcesPath().toString() + File.separatorChar + "keyfile.pfx");
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
KeyStore ks = KeyStore.getInstance("PKCS12");
InputStream kIn = new FileInputStream(f);
ks.load(kIn, certificatePassword.toCharArray());
kIn.close();
kmf.init(ks, certificatePassword.toCharArray());
You can add one pfx per certificate this way (and open the right one depeding on the user) or load all certificates into one pfx file, i'm not sure how to read a specific certificate from the file, but i'm sure stackoverflow can help there ;)