spark-shell上でcassandrarowを扱う
spark-shellの起動
cd spark-1.5.0
bin/spark-shell --jars spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar --conf spark.cassandra.connection.host = 127.0.0.1
cassandraTableの取得
import com.datastax.spark.connector._ //Imports basic rdd functions
import com.datastax.spark.connector.cql._ //(Optional) Imports java driver helper functions
val c = CassandraConnector(sc.getConf)
val d = sc.cassandraTable("test_from_spark", "fun");
scala> d.collect.foreach(println)
CassandraRow{k: 1, v: 10}
CassandraRow{k: 2, v: 20}
columnの選択、column dataの取得、where句でfilter
d.select("k").where("k = ?", 10).foreach(println)
CassandraRow(k: 10)
d.select("k", "v").where("k = ?", 12).map(row => row.get[Int]("v")).collect
res2: Array[Int] = Array(120)
cassandra tableへの書き込み
val d = [なんかRDD]
d.saveToCassandra("keyspace", "table", SomeColumns("col1", "col2", ....))
link
cd spark-1.5.0
bin/spark-shell --jars spark-cassandra-connector-assembly-1.5.0-M2-SNAPSHOT.jar --conf spark.cassandra.connection.host = 127.0.0.1
import com.datastax.spark.connector._ //Imports basic rdd functions
import com.datastax.spark.connector.cql._ //(Optional) Imports java driver helper functions
val c = CassandraConnector(sc.getConf)
val d = sc.cassandraTable("test_from_spark", "fun");
scala> d.collect.foreach(println)
CassandraRow{k: 1, v: 10}
CassandraRow{k: 2, v: 20}
columnの選択、column dataの取得、where句でfilter
d.select("k").where("k = ?", 10).foreach(println)
CassandraRow(k: 10)
d.select("k", "v").where("k = ?", 12).map(row => row.get[Int]("v")).collect
res2: Array[Int] = Array(120)
cassandra tableへの書き込み
val d = [なんかRDD]
d.saveToCassandra("keyspace", "table", SomeColumns("col1", "col2", ....))
link
d.select("k").where("k = ?", 10).foreach(println)
CassandraRow(k: 10)
d.select("k", "v").where("k = ?", 12).map(row => row.get[Int]("v")).collect
res2: Array[Int] = Array(120)
val d = [なんかRDD]
d.saveToCassandra("keyspace", "table", SomeColumns("col1", "col2", ....))
link
Author And Source
この問題について(spark-shell上でcassandrarowを扱う), 我々は、より多くの情報をここで見つけました https://qiita.com/rikima/items/73a6a7a9d7d284267301著者帰属：元の著者の情報は、元のURLに含まれています。著作権は原作者に属する。
Content is automatically searched and collected through network algorithms . If there is a violation . Please contact us . We will adjust (correct author information ,or delete content ) as soon as possible .