slides/02_scala.html

<!doctype html>
<!-- Inspired by view-source:https://brookewenig.github.io/SparkOverview.html  -->
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">

    <meta name="description" content="Apache Spark 2 Workshop | Scala &mdash; Just Enough For Developing Spark Applications">
    <meta name="author" content="Jacek Laskowski">

    <title>Apache Spark 2 Workshop | Scala &mdash; Just Enough For Developing Spark Applications</title>

    <link rel="stylesheet" href="reveal.js/css/reveal.css">
    <link rel="stylesheet" href="revealjs-css/jacek.css">
    <link rel="stylesheet" href="reveal.js/css/theme/beige.css">

    <!-- Theme used for syntax highlighting of code -->
    <link rel="stylesheet" href="reveal.js/lib/css/zenburn.css">

    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />

    <!-- Printing and PDF exports -->
    <script>
      var link = document.createElement( 'link' );
      link.rel = 'stylesheet';
      link.type = 'text/css';
      link.href = window.location.search.match( /print-pdf/gi ) ? 'reveal.js/css/print/pdf.css' : 'reveal.js/css/print/paper.css';
      document.getElementsByTagName( 'head' )[0].appendChild( link );
    </script>
  </head>

  <body>
    <div class="reveal">
      <div class="slides">

        <section class="intro" data-transition="zoom" id="home">
          <span class="menu-title" style="display: none">Home</span>
          <p>
            <img width="5%" style="background:none; border:none; box-shadow:none;" data-src="images/scala-logo.png">
            <img width="17%" style="background:none; border:none; box-shadow:none;" data-src="images/spark-logo.png">
            <img width="8%" src="images/jacek_laskowski_20141201_512px.png" style="border: 0">
          </p>
          <h1>Scala</h1>
          <h3>Just Enough For Developing Spark Applications</h3>

          <h4 style="font-size: smaller;"><a href="https://twitter.com/jaceklaskowski">@jaceklaskowski</a> / <a href="http://stackoverflow.com/users/1305344/jacek-laskowski">StackOverflow</a> / <a href="https://github.com/jaceklaskowski">GitHub</a><br />
          Books: <a href="http://bit.ly/mastering-apache-spark">Mastering Apache Spark</a> / <a href="http://bit.ly/mastering-spark-sql">Mastering Spark SQL</a> / <a href="http://bit.ly/spark-structured-streaming">Spark Structured Streaming</a></h4>

          <footer style="font-size: small;">&copy;<a href="https://medium.com/@jaceklaskowski">Jacek Laskowski</a> 2018 / @jaceklaskowski / jacek@japila.pl</footer>
        </section>

        <section id="agenda" data-markdown>
          <script type="text/template">
            <!-- .slide: style="font-size: 90%" -->
            ## Agenda

            1. [Introduction to Scala](#/intro)
            1. [Hello World Scala App](#/hello-world-app)
            1. [Spark Shell](#/spark-shell)
            1. [Scala Types](#/types)
            1. [Expressions](#/expressions)
            1. [Scaladoc](#/scaladoc)
            1. [Methods](#/methods)
            1. [Functions](#/functions)
            1. [for-comprehension](#/for-comprehension)
            1. [IntelliJ IDEA](#/intellij-idea)
            1. [ScalaTest](#/scalatest)
          </script>
        </section>

        <section>
          <section id="intro" data-markdown>
            <script type="text/template">
              ## Scala

              * General-Purpose Programming language
              * Strongly-Typed Language
                  * Types known at compile time
                  * Scala compiler as a safety net
                  * Application compiles? It then should run fine in production
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Scala

              * Functional and Object-Oriented Paradigms blended nicely
              * Class Hierarchies
              * Higher-Order Functions
              * The latest major version: **2.12**
              * Home page: <a href="http://www.scala-lang.org/">http://www.scala-lang.org</a>
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Scala and Spark

              * **Apache Spark** is written in **Scala**
              * Spark supports up to **2.11.12**
              * Support for 2.12 is coming
            </script>
          </section>
        </section>

        <section>
          <section id="hello-world-app" data-markdown>
            <script type="text/template">
              ## Hello World Scala App

```scala
  object Main extends App {
    println("Hello, Friends!")
  }

```

* Most likely the easiest Scala standalone application
  * You _could_ cut the number of lines as a Scala script
            </script>
          </section>
          <section id="object-keyword" data-markdown>
            <script type="text/template">
              ## object Keyword

```scala

// Creates a new type and the one and only value

scala> object Main
defined object Main

scala> object Jacek {
     |   def name = "Jacek"
     | }
defined object Jacek

scala> Jacek.name
res0: String = Jacek
```
            </script>
          </section>
          <section id="extends-keyword" data-markdown>
            <script type="text/template">
              ## extends Keyword

```scala

// Creates "is-a" relationship

scala> object Main extends App
defined object Main

scala> Main. // <-- press TAB after the dot
delayedInit   executionStart   main

// App type gives main method
// main is the entry point to Scala standalone applications

scala> Main.main(Array.empty)
```
            </script>
          </section>
          <section id="println" data-markdown>
            <script type="text/template">
              ## println Procedure

```scala

// Accepts one input parameter of type String
// Print the argument to the console

scala> println("Hello, Friends!")
Hello, Friends!
```
            </script>
          </section>
          <section id="curly-brackets">
            <h2>curly brackets</h2>
            <pre><code>
object Main extends App {
  println("Hello, Friends!")
}
            </code></pre>
          </section>
          <section>
            <h2>Hello World in Action</h2>
            <pre><code>
scala> :paste
// Entering paste mode (ctrl-D to finish)

object Main extends App {
  println("Hello, Friends!")
}

// Exiting paste mode, now interpreting.

defined object Main

scala> Main.main(null)
Hello, Friends!
            </code></pre>
          </section>
          <section>
            <h2>Hello World in Action</h2>
            <pre><code>
scala> :load yourScalaCode.txt
defined object Main
Hello, Friends!
            </code></pre>
          </section>
          <section id="val-keyword" data-markdown>
            <script type="text/template">
              ## val keyword

```scala

// Defines a value (aka constant)

scala> val name = "Jacek"
name: String = Jacek

scala> name
res0: String = Jacek
```
            </script>
          </section>
          <section id="string-interpolation" data-markdown>
            <script type="text/template">
              ## String Interpolation

```scala

// Replaces $-prefixed strings with corresponding values
// Allows for templating

scala> val name = "Jacek"
name: String = Jacek

// Note the 's' character before the text
//             |
//             v
scala> println(s"Hello, $name!")
Hello, Jacek!

// Template
scala> val greeting = s"Hello $name. Nice to meet you."
greeting: String = Hello Jacek. Nice to meet you.
```
            </script>
          </section>
          <section id="app-main-args" data-markdown>
            <script type="text/template">
              ## App, main and args

* **main** is the entry point to a Scala standalone application
* **App** gives **args** to access command-line arguments
  * **args** is of type **Array[String]**

```scala
scala> object Main extends App
defined object Main

scala> Main. // <-- press TAB after the dot
delayedInit   executionStart   main

// Execute main with one input argument

scala> Main.main(Array("Jacek"))
```
            </script>
          </section>
        </section>

        <section>
          <section id="spark-shell" data-markdown>
            # spark-shell
            ### Learning Scala (and Spark) Interactively
          </section>
          <section data-markdown>
            <script type="text/template">
              ## spark-shell

* **Scala REPL** is an interactive environment for Scala exploration and prototyping
  * Extremely helpful during your first days in Scala
* **spark-shell** is Scala REPL with Scala imports and values
  * Gives you both worlds — Scala and Spark

```
$ ./bin/spark-shell
```
            </script>
          </section>
          <section data-markdown>
            ## Enjoy Scala (Exercises)

            1. Get familiar with type literals
              * Strings, Ints, Longs, Doubles, Chars, Symbols
            1. **Println** your name to the console
            1. Define your name as a **val** and use to println
              * Use string interpolation #protip
            1. Save the code to a file and **:load** it
          </section>
        </section>

        <section>
          <section id="types" data-markdown>
            # Scala Types
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Built-In "Primitive" Types

              * Numeric types
                * Int, Long, Double
              * Character types
                * Char, Symbol
                * String
              * Logical type
                * Boolean
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Standard Types

              * Tuple **(1,2)**
              * Seq(uence) **Seq(0,1,2,3)**
              * Range **0 to 4**
              * Iterator **Seq(...).toIterator**
              * Function
                * <!-- .element: style="font-size: 0.8em;" --> _More in the upcoming [slide](#/standard-functions-datetime)_
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Custom Types

              * **case class** Keywords
                * Auto-generates many methods
              * **class** Keyword
                * Less often in Spark

```
// Define a custom type
scala> case class Person(name: String, age: Int)
defined class Person

// Create an instance
scala> val jacek = Person("Jacek", 44)
jacek: Person = Person(Jacek,44)
```
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Types in spark-shell

```
scala> :help type

display the type of an expression without evaluating it

scala> val lang = "Scala"
lang: String = Scala

scala> :type lang
String

// Object defined previously
scala> :type jacek
Person
```
            </script>
          </section>
        </section>

        <section>
          <section id="expressions" data-markdown>
            # Expressions

            ### ... And Statements
          </section>
          <section data-markdown>
            <script type="text/template">
              ## Expression

              * Unit of code that returns a value
              * **Expression block** is a collection of expressions wrapped in curly braces

```scala
scala> val five = 5
five: Int = 5

scala> five + 10
res1: Int = 15
```

```scala
// What's the value of block value?
// Use spark-shell to answer the question.
val block = {
  "hello world"
  5
}
```
<!-- .element: class="fragment" -->
            </script>
          </section>
          <section data-markdown>
            <script type="text/template">
              ## If Expression

              * **if (boolean expression) expr1 else expr2**

```scala
scala> val five = if (true) 5 else 10
five: Int = 5

scala> val ten = if (false) 5 else 10
ten: Int = 10
```
<!-- .element: class="fragment" -->
            </script>
          </section>
        </section>

        <section>
          <section id="scaladoc" data-markdown>
            # Scaladoc
            ## Scala Documentation
          </section>
          <section data-markdown>
            ## Scala Standard Library

            * https://www.scala-lang.org/api/2.11.12
            * Spend as much time with the docs as possible
              * That will pay off quickly
          </section>
        </section>

        <section>
          <section data-markdown>
            # Methods in Scala
            ## Named executable blocks of code
          </section>
          <section data-markdown>
            ## Method Definition

            * **def** Keyword
            * Optional input parameters (in round brackets)
              * **name: Type**
              * comma as separator for multiple input params
            * Optional return type

```
scala> def greet(name: String): String = s"Hello, $name"
greet: (name: String)String

scala> greet("Jacek")
res8: String = Hello, Jacek

// What's res8?
```

          </section>
          <section id="methods" data-markdown>
            ## Methods

            * Methods belong to classes
              * Class as a container of methods (and state)
            * Implicit **this** parameter for the owning object

```
case class Person(name: String) {
  def introduceYourself: String = s"I'm $name"
}

// Use :paste or :load

scala> val jacek = new Person("Jacek")
jacek: Person = Person(Jacek)

scala> jacek.introduceYourself
res9: String = I'm Jacek
```

          </section>
          <section data-markdown>
            ## Methods in spark-shell

            * spark-shell auto-generates an implicit object
              * Another reason to use spark-shell early and often
              * Makes for a better learning experience

```
scala> def greet(name: String) = s"Hello, $name"
greet: (name: String)String

scala> greet("Jacek")
res8: String = Hello, Jacek
```

          </section>
          <section data-markdown>
            ## Named Parameters

            * Use parameter names at call site
            * Any order of parameters possible

```
scala> def greet(name: String) = s"Hello, $name"
greet: (name: String)String

scala> greet(name = "Jacek")
res8: String = Hello, Jacek
```

          </section>
        </section>

        <section>
          <section id="functions" data-markdown>
            # Functions in Scala
            ## (Other) Named executable blocks
          </section>
          <section data-markdown>
            ## Function Definition

            * Functions are values of a function type
            * Part of objects or classes

```
scala> val greet: String => String = { name => s"Hello, $name" }
greet: String => String = <function1>

scala> val greet = { name: String => s"Hello, $name" }
greet: String => String = <function1>

scala> greet("Jacek")
res10: String = Hello, Jacek
```

          </section>
          <section data-markdown>
            ## _ Placeholder

            * **_** (underscore) to reference one input parameter
            * \# underscores = \# input parameters
            * Access ordered from left to right
            * Very useful notation with higher-order functions
              * Higher-order functions defined on the following slide

```
// What's the more proper name of the function?
scala> val f: (Int, Int) => Int = _ + _
f: (Int, Int) => Int = <function2>
```

          </section>
        </section>

        <section>
          <section id="for-comprehension" data-markdown>
            # for-comprehension
            ## Functional for Loop
          </section>
          <section data-markdown>
            ## Demo

            * Generating Coordinates (as Pairs of Numbers)
          </section>
        </section>

        <section id="intellij-idea" data-markdown>
          # IntelliJ IDEA

          ## Integrated development environment
          ## for Scala and Spark
        </section>

        <section>
          <section>
            <h2>IntelliJ IDEA</h2>
            <ol>
              <li>
                <b>Integrated development environment</b>
                <ul>
                  <li>The best development tool for Scala</li>
                </ul>
              </li>
              <li>Support for Scala using separate <b>Scala plugin</b></li>
              <li><b>Scala Worksheet</b> for Learning and Prototyping</li>
              <li>"IntelliSense" Feature</li>
              <li>Toggle <b>Distraction Free mode</b></li>
              <li>
                Lots of Key Shortcuts
                <ul>
                  <li>You're going to use them quite often!</li>
                </ul>
              </li>
            </ol>
          </section>
          <section>
            <h2>IntelliJ IDEA &mdash; Demo</h2>
            <ol>
              <li>
                Creating a Scala sbt-managed project
                <ul>
                  <li>Enable <b>Use auto-import</b> checkbox</li>
                  <li>Disable <b>Create directories for empty content roots automatically</b> checkbox</li>
                  <li>Wait till IntelliJ IDEA (and sbt) resolves the project's dependencies that may take some <i>longer</i> time</li>
                </ul>
              </li>
              <li>
                Running the Hello World application
                <ul>
                  <li>Right-click the object name and select <b>Run</b></li>
                  <li>Click <b>double-window icon</b> on the stripe on the left</li>
                </ul>
              </li>
            </ol>
          </section>
          <section>
            <h2>IntelliJ IDEA &mdash; Exercise</h2>
            <ol>
              <li>
                Create a Scala standalone application using IntelliJ IDEA
                <ul>
                  <li>Use a Scala sbt-managed project</li>
                </ul>
              </li>
              <li>
                Run the Scala application
              </li>
              <li>Pass command-line arguments
                <ul>
                  <li><b>Run > Edit Configurations</b> menu</li>
                  <li><b>Program arguments</b> field</li>
                  <li>Run the application once to have an configuration entry</li>
                </ul>
              </li>
            </ol>
          </section>
        </section>

        <section id="scalatest">
          <h1>ScalaTest</h1>
        </section>

        <section>
          <section>
            <h2>ScalaTest</h2>
            <ol>
              <li>
                Scala Testing Framework
              </li>
              <li>
                Matchers - Assertions
              </li>
              <li>
                sbt supports ScalaTest
                <ul>
                  <li><b>sbt test</b> (and don't forget about <b>~</b> - a tilde)</li>
                </ul>
              </li>
              <li>
                IntelliJ IDEA supports ScalaTest, too.
              </li>
              <li><a href="http://www.scalatest.org">http://www.scalatest.org</a></li>
              <li><b>Demo:</b> Installation and Execution</li>
            </ol>
          </section>
          <section>
            <h2>Exercise: Writing a Test</h2>
            <ol>
              <li>
                Follow <a href="http://www.scalatest.org">http://www.scalatest.org</a>
              </li>
              <li>
                Define <b>libraryDependencies</b> in <b>build.sbt</b>
              </li>
              <li>
                Run tests using <b>sbt test</b>
              </li>
            </ol>
          </section>
        </section>

        <section id="recap" data-markdown>
          <script type="text/template">
            <!-- .slide: style="font-size: 90%" -->
            ## Recap

            1. [Introduction to Scala](#/intro)
            1. [Hello World Scala App](#/hello-world-app)
            1. [Spark Shell](#/spark-shell)
            1. [Scala Types](#/types)
            1. [Expressions](#/expressions)
            1. [Scaladoc](#/scaladoc)
            1. [Methods](#/methods)
            1. [Functions](#/functions)
            1. [for-comprehension](#/for-comprehension)
            1. [IntelliJ IDEA](#/intellij-idea)
            1. [ScalaTest](#/scalatest)
          </script>
        </section>

        <section style="text-align: left" data-markdown id="questions">
          <script type="text/template">
            # Questions?

            * Read [Mastering Apache Spark 2](https://bit.ly/mastering-apache-spark)
              * [https://bit.ly/mastering-apache-spark](https://bit.ly/mastering-apache-spark)
            * Read [Mastering Spark SQL](https://bit.ly/mastering-spark-sql)
              * [https://bit.ly/mastering-spark-sql](https://bit.ly/mastering-spark-sql)
            * Follow [@jaceklaskowski](https://twitter.com/jaceklaskowski) on twitter
            * Upvote [my questions and answers on StackOverflow](http://stackoverflow.com/users/1305344/jacek-laskowski)
          </script>
        </section>

      </div>
    </div>

    <script src="reveal.js/lib/js/head.min.js"></script>
    <script src="reveal.js/js/reveal.js"></script>

    <script>
      // More info about config & dependencies:
      // - https://github.com/hakimel/reveal.js#configuration
      // - https://github.com/hakimel/reveal.js#dependencies
      Reveal.initialize({
        controls: true,
        progress: true,
        history: true,
        center: true,
        slideNumber: true,

        transition: 'slide', // none/fade/slide/convex/concave/zoom

        menu: {
          markers: true,
          openSlideNumber: true
        },
        dependencies: [
          { src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
          { src: 'reveal.js/plugin/markdown/marked.js' },
          { src: 'reveal.js/plugin/markdown/markdown.js' },
          { src: 'reveal.js/plugin/zoom-js/zoom.js', async: true },
          { src: 'reveal.js/plugin/notes/notes.js', async: true },
          { src: 'revealjs-plugins/menu/menu.js', async: true },
          { src: 'reveal.js/plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } }
        ]
      });
    </script>
    <script>
      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
      (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
      m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
      })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

      ga('create', 'UA-45999426-3', 'auto');
      ga('send', 'pageview');

    </script>
  </body>
</html>