Serverless Image Classification with Oracle Functions and TensorFlow

Image classification is a canonical example used to demonstrate machine learning techniques. This post shows you how to run a TensorFlow-based image classification application on the recently announced cloud service Oracle Functions.

Photo by Franck V. on Unsplash

Oracle Functions

Oracle Functions is powered by the Fn Project, which is an open source, container native, serverless platform that can be run anywhere — in any cloud or on-premises. You can download and install the open source distribution of Fn Project, develop and test a function locally, and then use the same tooling to deploy that function to Oracle Functions.

What to Expect

For example, when passed to the classification function, this image returned — This is a ‘pizza’ Accuracy — 100%.

Photo by Alan Hardman on Unsplash

The Code

Function Image Input

  • Simple data binding, like handling string input.
  • Binding JSON data types to POJOs. You can customize this because it’s internally implemented using Jackson.
  • Working with raw inputs, enabled by an abstraction of the raw Fn Java FDK events received or returned by the function.

The binding can be further extended if you want to customize the way your input and output data is marshaled.

The existing TensorFlow example expects a list of image names (which must be present on the machine from which the code is being executed) as input. The function behaves similarly, but with an important difference — it uses the flexible binding capability provided by the Fn Java FDK. The classify method serves as the entry point to the function and accepts a Java byte array (byte[]), which represents the raw bytes of the image that is passed into the function. This byte array is then used to create the Tensor object using the static Tensor.create(byte[]) method:

public class LabelImageFunction { 
public String classify(byte[] image) { ...
Tensor<String> input = Tensors.create(image); ...
}
}

The full source code is available on GitHub.

Machine Learning Model

  • Training: An algorithm is fed with past (historical) data in order to learn from it (derive patterns) and build a model. Very often, this process is ongoing.
  • Predicting: The generated model is then used to generate predictions or outputs in response to new inputs based on the facts that were learned during the training phase

This application uses a pregenerated model. As an added convenience, the model (and labels) required by the classification logic are packaged with the function itself (part of the Docker image). These can be found in the resources folder of the source code.

This means that you don’t have to set up a dedicated model serving component (like TensorFlow Serving).

Function Metadata

schema_version: 20180708 
name: classify
version: 0.0.1
runtime: java
memory: 1024
timeout: 120
triggers:
- name: classify
type: http
source: /classify

Here is a summary of the attributes used:

  • schema_version represents the version of the specification for this file.
  • name is the name and tag to which this function is pushed.
  • version represents the current version of the function. When deploying, it is appended to the image as a tag.
  • runtime represents the programming language runtime, which is java in this case.
  • memory (optional) is the maximum memory threshold for this function. If this function exceeds this limit during execution, it’s stopped and an error message is logged.
  • timeout (optional) is the maximum time that a function is allowed to run.
  • triggers (optional) is an array of trigger entities that specify triggers for the function. In this case, we’re using an HTTP trigger.

Function Dockerfile

Here is the default Dockerfile that is used to create Docker images for your functions:

FROM fnproject/fn-java-fdk-build:jdk9-1.0.75 as build-stage 
WORKDIR /function
ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository
ADD pom.xml /function/pom.xml
RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"]
ADD src /function/src
RUN ["mvn", "package"]
FROM fnproject/fn-java-fdk:jdk9-1.0.75
WORKDIR /function
COPY --from=build-stage /function/target/*.jar /function/app/
CMD ["com.example.fn.HelloFunction::handleRequest"]

It’s a multiple-stage Docker build that performs the following actions (out-of-the-box):

  • Maven package and build
  • Copying (using COPY) the function JAR and dependencies to the runtime image
  • Setting the command to be executed (using CMD) when the function container is spawned

But there are times when you need more control over the creation of the Docker image, for example, to incorporate native third-party libraries. In such cases, you want to use a custom Dockerfile. It’s powerful because it gives you the freedom to define the recipe for your function. All you need to do is extend from the base Docker images.

Following is the Dockerfile used for this function:

FROM fnproject/fn-java-fdk-build:jdk9-1.0.75 as build-stage 
WORKDIR /function
ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository ADD pom.xml /function/pom.xml
RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"]'
ARG TENSORFLOW_VERSION=1.12.0
RUN echo "using tensorflow version " $TENSORFLOW_VERSION RUN curl -LJO https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-$TENSORFLOW_VERSION.jar
RUN curl -LJO https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-cpu-linux-x86_64-$TENSORFLOW_VERSION.tar.gz RUN tar -xvzf libtensorflow_jni-cpu-linux-x86_64-$TENSORFLOW_VERSION.tar.gz
ADD src /function/src
RUN ["mvn", "package"]
FROM fnproject/fn-java-fdk:jdk9-1.0.75
ARG TENSORFLOW_VERSION=1.12.0
WORKDIR /function
COPY --from=build-stage /function/libtensorflow_jni.so /function/runtime/lib
COPY --from=build-stage /function/libtensorflow_framework.so /function/runtime/lib
COPY --from=build-stage /function/libtensorflow-$TENSORFLOW_VERSION.jar /function/app/
COPY --from=build-stage /function/target/*.jar /function/app/
CMD ["com.example.fn.LabelImageFunction::classify"]

Notice the additional customization that it incorporates, in addition to the default steps like Maven build:

  • Automates TensorFlow setup (per the instructions), extracts the TensorFlow Java SDK and the native JNI (.so) libraries
  • (as part of the second stage of the Docker build) Copies the JNI libraries to /function/runtime/lib and the SDK JAR to /function/app so that they are available to the function at runtime

Deploying to Oracle Functions

curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

​You can also download it directly from https://github.com/fnproject/cli/releases.

Oracle Functions Context

When the Fn Project CLI is initially installed, it’s configured for a local development context. To configure the Fn Project CLI to connect to your Oracle Cloud Infrastructure tenancy instead, you have to create a new context. The context information is stored in a .yaml file in the ~/.fn/contexts directory. It specifies Oracle Functions endpoints, the OCID of the compartment to which deployed functions belong, the Oracle Cloud Infrastructure configuration file, and the address of the Docker registry to push images to and pull images from.

This is what a context file looks like:

api-url: https://functions.us-phoenix-1.oraclecloud.com oracle.compartment-id: <OCI_compartment_OCID> 
oracle.profile: <profile_name_in_OCI_config>
provider: oracle
registry: <OCI_docker_registry>

Oracle Cloud Infrastructure Configuration

Here is an example configuration file:

[DEFAULT] 
user=ocid1.user.oc1..exampleuniqueID fingerprint=20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34 key_file=~/.oci/oci_api_key.pem tenancy=ocid1.tenancy.oc1..exampleuniqueID pass_phrase=tops3cr3t region=us-ashburn-1
[ORACLE_FUNCTIONS_USER]
user=ocid1.user.oc1..exampleuniqueID fingerprint=72:00:22:7f:d3:8b:47:a4:58:05:b8:95:84:31:dd:0e key_file=/.oci/admin_key.pem tenancy=ocid1.tenancy.oc1..exampleuniqueID pass_phrase=s3cr3t region=us-phoenix-1

You can define multiple contexts, each stored in a different context file. Switch to the correct context according to your Functions development environment:

fn use context <context_name>

Create the Application

git clone https://github.com/abhirockzz/fn-hello-tensorflow

Here is the command required to deploy an application:

fn create app <app_name> --annotation oracle.com/oci/subnetIds='["<subnet_ocid>"]'
  • <app_name> is the name of the new application.
  • <subnet_ocid> is the OCID of the subnet in which to run your function.

For example:

fn create app fn-tensorflow-app --annotation oracle.com/oci/subnetIds='["ocid1.subnet.oc1.phx.exampleuniqueID","ocid1.subnet.oc1.phx.exampleuniqueID","ocid1.subnet.oc1.phx.exampleuniqueID"]'

Deploy the Function

fn deploy --app <app_name>

<app_name> is the name of the application in Oracle Functions to which you want to add the function.

If you want to use TensorFlow version 1.12.0 (for Java SDK and corresponding native libraries), use the following command:

fn -v deploy --app fn-tensorflow-app

You can also choose a specific version. Ensure that you specify it in pom.xml file before you build the function. For example, if you want to use version 1.11.0:

<dependency> 
<groupId>org.tensorflow</groupId> <artifactId>tensorflow</artifactId>
<version>1.11.0</version>
<scope>provided</scope>
</dependency>

To specify the version during function deployment, you can use — build-arg (build argument) as follows:

fn -v deploy --app fn-tensorflow-app --build-arg TENSORFLOW_VERSION=<version>

For example, if you want to use 1.11.0:

fn -v deploy --app fn-tensorflow-app --build-arg TENSORFLOW_VERSION=1.11.0

When the deployment completes successfully, your function is ready to use. Use the fn ls apps command to list down the applications currently deployed. fn-tensorflow-app should be listed.

Time to Classify Images!

You can start by downloading some of the recommended images or use images that you already have on your computer. All you need to do is pass them to the function while invoking it:

cat <path to image> | fn invoke fn-tensorflow-app classify

Ok, let’s try this. Can it detect the sombrero in this image?

cat /Users/abhishek/manwithhat.jpg | fn invoke fn-tensorflow-app classify
“366 • 9 • Gringo” (CC BY-NC-ND 2.0) by Pragmagraphr

Result:

This is a ‘sombrero’ Accuracy — 92%

How about a terrier?

cat /Users/abhishek/terrier.jpg | fn invoke fn-tensorflow-app classify
“Terrier” (CC BY-NC 2.0) by No_Water

Result:

This is a 'West Highland white terrier' Accuracy - 88%

What will you classify? :-)

Summary

Oracle Functions will be generally available in 2019, but we are currently providing access to selected customers through our Cloud Native Limited Availability Program. To learn more about Oracle Functions or to request access, please register. You can also learn more about the underlying open source technology used in Oracle Functions at FnProject.io.

Originally published at blogs.oracle.com on January 7, 2019.

Currently working with Kafka, Databases, Azure, Kubernetes and related open source projects | Confluent Community Catalyst (for Kafka)