spark streaming batch size spark streaming kafka에서 batch size 조절 옵션 df = ( stream_reader .option("maxOffsetsPerTrigger", "100000") .load() ) https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html Data Engineering 2023.07.21