Apache Spark integration with mendix

0
In Our project we are using database connector module to fetch the data from datalake via execute query but its taking too much of time inorder to avoid the time constraint I wants to replace that with spark sql so Is there any possibilities to integrate apache spark sql with mendix. As of now i am trying with apache spark version 3.1.1 and Eclipse latest version i am facing some issues in the code so, suggest me what an all the dependencies required to integrate spark with mendix. Also please advice me like Is there any other way to fetch the data from datalake faster than the database connector. I am facing these kinds of issues kindly advice me on this       Thanks In Advance, Shahistha
asked
3 answers
2

You need these lib in order to build java action base on RDD programming.

import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.SparkConf;
answered
1

According to the documentation, you should be able to connect to spark using Jdbc. More info on this here

I think your best bet is to use the database connector to connect to spark sql. 

answered
0

Hi , Are you able to Integrate Apache Spark with mendix ? Can you please let me know . Even in my project , i am planning to use database connector for Hadoop data lake along with Apache Spark , so if with java action are you able to integrate Spark with Mendix ?

answered