error: missing or invalid dependency detected while loading class file 'RDD.class'.

問題は次のように再現されます.

error: missing or invalid dependency detected while loading class file 'RDD.class'.
Could not access term hadoop in package org.apache,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDD.class' was compiled against an incompatible version of org.apache.
error: missing or invalid dependency detected while loading class file 'RDD.class'.
Could not access term io in value org.apache.hadoop,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDD.class' was compiled against an incompatible version of org.apache.hadoop.
warning: Class com.fasterxml.jackson.annotation.JsonInclude not found - continuing with a stub.
warning: Class com.fasterxml.jackson.annotation.JsonInclude not found - continuing with a stub.
warning: While parsing annotations in self_jars/spark-core_2.11-2.3.1.jar(org/apache/spark/rdd/RDDOperationScope.class), could not find NON_NULL in enum .
This is likely due to an implementation restriction: an annotation argument cannot refer to a member of the annotated class (SI-7014).
warning: Class com.fasterxml.jackson.annotation.JsonPropertyOrder not found - continuing with a stub.
error: missing or invalid dependency detected while loading class file 'RDDOperationScope.class'.
Could not access term fasterxml in package com,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDDOperationScope.class' was compiled against an incompatible version of com.
error: missing or invalid dependency detected while loading class file 'RDDOperationScope.class'.
Could not access term jackson in value com.fasterxml,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDDOperationScope.class' was compiled against an incompatible version of com.fasterxml.
error: missing or invalid dependency detected while loading class file 'RDDOperationScope.class'.
Could not access term annotation in value com.jackson,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDDOperationScope.class' was compiled against an incompatible version of com.jackson.
error: missing or invalid dependency detected while loading class file 'RDDOperationScope.class'.
Could not access term JsonInclude in value com.annotation,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'RDDOperationScope.class' was compiled against an incompatible version of com.annotation.
four warnings found
6 errors found

hadoop-common-2.7.3をインポートする.jarでいい
scalacのインポート方法はclasspathの後ろのパスに対応するフォルダに上記のjarパッケージを入れることです.例えば、次のようにします.
scalac -classpath $(echo *.jar self_jars/*.jar | tr ' ' ':')  TestRdd.scala