You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1234567891011121314
  1. SPARK_HOME="/home/osboxes/Downloads/spark-3.2.2-bin-hadoop3.2"
  2. MEMORY="1g"
  3. SPARK_MASTER="spark://osboxes:7077"
  4. CASSANDRA_HOST="localhost"
  5. "$SPARK_HOME"/bin/spark-submit \
  6. --master "$SPARK_MASTER" \
  7. --conf spark.executor.memory="$MEMORY" \
  8. --conf spark.cassandra.connection.host="$CASSANDRA_HOST" \
  9. --conf spark.sql.session.timeZone=UTC \
  10. --conf spark.sql.extensions=com.datastax.spark.connector.CassandraSparkExtensions \
  11. --packages com.datastax.spark:spark-cassandra-connector_2.12:3.2.0 \
  12. ./src/spark/main.py