Business rules should have to do with aspects of business. An example of this is when a customer should be considered as a VIP customer: A customer which places more than 10 orders per month is a VIP customer.

This opposed to system rules which have to do with lower level functionality, such as whether an import of an excel file has already been performed.

As a side note, business rules in the Modeler currently always refer to structural business rules. Structural business rules always define some aspect of business (such as the VIP customer example above).

Besides structural business rules also operative business rules exist. These are rules which constrain some aspect of business. An example of this is: A VIP customer gets 10% discount on every order he/she places. These rules are only implicit at the moment, for example by creating a microflow which contains a gateway with a structural business rule to define whether a customer is a VIP customer and subsequently a change activity which applies the discount

I think system rules are decision rules for system-related decisions. E.g. Where a business rules determines the rule to set a priority for a delivery, a system rule determines when user will receive a update for his password.