This repository was archived by the owner on Jun 14, 2024. It is now read-only.
File tree 1 file changed +59
-13
lines changed
1 file changed +59
-13
lines changed Original file line number Diff line number Diff line change @@ -239,16 +239,62 @@ hs.explain(query, verbose = True)
239
239
240
240
#### Enable Hyperspace
241
241
242
- Now that you have created an index that your query can utilize, you can enable Hyperspace and execute your query:
243
-
244
- Scala:
245
- ``` scala
246
- spark.enableHyperspace
247
- query.show
248
- ```
249
-
250
- Python:
251
- ``` python
252
- Hyperspace.enable(spark)
253
- query.show()
254
- ```
242
+ Now that you have created an index that your query can utilize, you can enable Hyperspace and execute your query.
243
+ There are two ways to enable Hyperspace:
244
+
245
+ 1 . Using Hyperspace extension
246
+
247
+ You can start a session with Hyperspace extension.
248
+ The follows are examples to use Spark™ with Hyperspace.
249
+
250
+ spark-submit:
251
+ ```
252
+ spark-submit -c spark.sql.extensions=com.microsoft.hyperspace.HyperspaceSparkSessionExtension ...
253
+ ```
254
+
255
+ Scala:
256
+ ```scala
257
+ val spark = SparkSession
258
+ .builder()
259
+ .appName("...")
260
+ .master("...")
261
+ .config("spark.sql.extensions", "com.microsoft.hyperspace.HyperspaceSparkSessionExtension")
262
+ .getOrCreate()
263
+ ```
264
+
265
+ Python:
266
+ ```python
267
+ from pyspark.sql import SparkSession
268
+
269
+ spark = SparkSession \
270
+ .builder \
271
+ .appName("...") \
272
+ .master("...") \
273
+ .config("spark.sql.extensions", "com.microsoft.hyperspace.HyperspaceSparkSessionExtension") \
274
+ .getOrCreate()
275
+ ```
276
+
277
+ Spark configuration file (default: `$SPARK_HOME/conf/spark-defaults.conf`):
278
+
279
+ ```txt
280
+ # Add the following line to the configuration file.
281
+
282
+ spark.sql.extensions com.microsoft.hyperspace.HyperspaceSparkSessionExtension
283
+ ```
284
+
285
+
286
+ 2. Calling enableHyperspace function explicitly
287
+
288
+ By explicitly calling enableHyperspace (for Scala) or Hyperspace.enable (for Python), Spark will use Hyperspace indexes when it is applicable.
289
+
290
+ Scala:
291
+ ```scala
292
+ spark.enableHyperspace
293
+ query.show
294
+ ```
295
+
296
+ Python:
297
+ ```python
298
+ Hyperspace.enable(spark)
299
+ query.show()
300
+ ```
You can’t perform that action at this time.
0 commit comments