According to Google, your application is well below the maximum file size, which is 100MB without expansion files (source). It might be that older phones have a smaller cache partition so your app doesn't fit, hence the increase in supported devices after removing the crosswalk plug-in.
The question is how many people your app can currently reach, the number of supported devices doesn't tell you that. The market share of the supported devices is much more interesting.
We've added Crosswalk to be able to offer a consistent experience across Android devices. It also makes debugging problems a lot easier if you know that all Android users are using the same browser and browser version. If I remember correctly Crosswalk also requires a newer Android version than the other plug-ins we ship by default, that would also decrease the number of supported devices.
The solution was found in this thread by Paul Ketelaars. The right architecture was not targeted (ARM vs x86), which was solved using this plugin.