Hi Priya, the REST api can output a dataset (list). This needs to be the output of the microflow of your operation. If you return a List of Objects, that will be the output.
If you expose a list you need to configure a Export Mapping.
See for more information on Operations: https://docs.mendix.com/refguide/published-rest-operation/
I would like to suggest you follow this learning path to run you through the integration basics and has a step by step approach for setting up a REST service (both publish and consume).
If you publish a REST service from an entity, you should get the option to specify the type of GET (All and by Key). This will create a corresponding microflow which is triggered when calling the correct service. By default this will be <appurl>/rest/<servicename>/v1/<entityname> for get all.
In your "publish app", you can also use Swagger UI to test it. See test section of the above mentioned learning path.
Update: Microflow calling the rest service
Note: Have to check why the second action has (Not set) for the entity. Could be a mx 10.6.1 issue...
And the simple domain model
In the published service you need to make sure done the following steps: