Serverless Image Classification with Oracle Functions and TensorFlow

Image classification is a canonical example used to demonstrate machine learning techniques. This post shows you how to run a TensorFlow-based image classification application on the recently announced cloud service Oracle Functions.

Photo by Franck V. on Unsplash

Oracle Functions

Oracle Functions which is a fully managed, highly scalable, on-demand, function-as-a-service platform built on enterprise-grade Oracle Cloud Infrastructure. It’s a serverless offering that enables you to focus on writing code to meet business needs without worrying about the underlying infrastructure, and get billed only for the resources consumed during the execution. You can deploy your code and call it directly or in response to triggers — Oracle Functions does all the work required to ensure that your application is highly available, scalable, secure, and monitored.

Oracle Functions is powered by the Fn Project, which is an open source, container native, serverless platform that can be run anywhere — in any cloud or on-premises. You can download and install the open source distribution of Fn Project, develop and test a function locally, and then use the same tooling to deploy that function to Oracle Functions.

What to Expect

Before we dive into the details, let’s see what you can expect from your serverless machine learning function. After it’s set up and running, you can point the app to images and it will return an estimate of what it thinks the image is, along with the accuracy of the estimate.

For example, when passed to the classification function, this image returned — This is a ‘pizza’ Accuracy — 100%.

Photo by Alan Hardman on Unsplash

The Code

The image classification function is based on an existing TensorFlow example. It leverages the TensorFlow Java SDK, which in turn uses the native C++ implementation using JNI (Java Native Interface).

Function Image Input

The image classification function leverages the Fn Java FDK, which simplifies the process of developing and running Java functions. One of its benefits is that it can seamlessly convert the input sent to your functions into Java objects and types. This includes:

• Simple data binding, like handling string input.
• Binding JSON data types to POJOs. You can customize this because it’s internally implemented using Jackson.
• Working with raw inputs, enabled by an abstraction of the raw Fn Java FDK events received or returned by the function.

The binding can be further extended if you want to customize the way your input and output data is marshaled.

The existing TensorFlow example expects a list of image names (which must be present on the machine from which the code is being executed) as input. The function behaves similarly, but with an important difference — it uses the flexible binding capability provided by the Fn Java FDK. The classify method serves as the entry point to the function and accepts a Java byte array (byte[]), which represents the raw bytes of the image that is passed into the function. This byte array is then used to create the Tensor object using the static Tensor.create(byte[]) method:

public class LabelImageFunction { 
  public String classify(byte[] image) { ... 
   Tensor<String> input = Tensors.create(image); ... 
   } 
}

The full source code is available on GitHub.

Machine Learning Model

Typically, a machine-learning-based system consists of the following phases:

• Training: An algorithm is fed with past (historical) data in order to learn from it (derive patterns) and build a model. Very often, this process is ongoing.
• Predicting: The generated model is then used to generate predictions or outputs in response to new inputs based on the facts that were learned during the training phase

This application uses a pregenerated model. As an added convenience, the model (and labels) required by the classification logic are packaged with the function itself (part of the Docker image). These can be found in the resources folder of the source code.

This means that you don’t have to set up a dedicated model serving component (like TensorFlow Serving).

Function Metadata

The func.yaml file contains function metadata, including attributes like memory and timeout (for this function, they are 1024 MB and 120 seconds, respectively). This metadata is required because of the (fairly) demanding nature of the image classification algorithm (as opposed to simpler computations).

schema_version: 20180708 
name: classify 
version: 0.0.1 
runtime: java 
memory: 1024 
timeout: 120 
triggers: 
 - name: classify 
   type: http 
   source: /classify

Here is a summary of the attributes used:

• schema_version represents the version of the specification for this file.
• name is the name and tag to which this function is pushed.
• version represents the current version of the function. When deploying, it is appended to the image as a tag.
• runtime represents the programming language runtime, which is java in this case.
• memory (optional) is the maximum memory threshold for this function. If this function exceeds this limit during execution, it’s stopped and an error message is logged.
• timeout (optional) is the maximum time that a function is allowed to run.
• triggers (optional) is an array of trigger entities that specify triggers for the function. In this case, we’re using an HTTP trigger.

Function Dockerfile

Oracle Functions uses a set of prebuilt, language-specific Docker images for build and runtime phases. For example, for Java functions, fn-java-fdk-build is used for the build phase and fn-java-fdk is used at runtime.

Here is the default Dockerfile that is used to create Docker images for your functions:

FROM fnproject/fn-java-fdk-build:jdk9-1.0.75 as build-stage 
WORKDIR /function 
ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository 
ADD pom.xml /function/pom.xml 
RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"] 
ADD src /function/src 
RUN ["mvn", "package"] 
FROM fnproject/fn-java-fdk:jdk9-1.0.75 
WORKDIR /function 
COPY --from=build-stage /function/target/*.jar /function/app/ 
CMD ["com.example.fn.HelloFunction::handleRequest"]

It’s a multiple-stage Docker build that performs the following actions (out-of-the-box):

• Maven package and build
• Copying (using COPY) the function JAR and dependencies to the runtime image
• Setting the command to be executed (using CMD) when the function container is spawned

But there are times when you need more control over the creation of the Docker image, for example, to incorporate native third-party libraries. In such cases, you want to use a custom Dockerfile. It’s powerful because it gives you the freedom to define the recipe for your function. All you need to do is extend from the base Docker images.

Following is the Dockerfile used for this function:

FROM fnproject/fn-java-fdk-build:jdk9-1.0.75 as build-stage 
WORKDIR /function 
ENV MAVEN_OPTS -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -Dhttp.nonProxyHosts= -Dmaven.repo.local=/usr/share/maven/ref/repository ADD pom.xml /function/pom.xml 
RUN ["mvn", "package", "dependency:copy-dependencies", "-DincludeScope=runtime", "-DskipTests=true", "-Dmdep.prependGroupId=true", "-DoutputDirectory=target", "--fail-never"]' 
ARG TENSORFLOW_VERSION=1.12.0 
RUN echo "using tensorflow version " $TENSORFLOW_VERSION RUN curl -LJO https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-$TENSORFLOW_VERSION.jar 
RUN curl -LJO https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-cpu-linux-x86_64-$TENSORFLOW_VERSION.tar.gz RUN tar -xvzf libtensorflow_jni-cpu-linux-x86_64-$TENSORFLOW_VERSION.tar.gz 
ADD src /function/src 
RUN ["mvn", "package"]
FROM fnproject/fn-java-fdk:jdk9-1.0.75 
ARG TENSORFLOW_VERSION=1.12.0 
WORKDIR /function 
COPY --from=build-stage /function/libtensorflow_jni.so /function/runtime/lib 
COPY --from=build-stage /function/libtensorflow_framework.so /function/runtime/lib 
COPY --from=build-stage /function/libtensorflow-$TENSORFLOW_VERSION.jar /function/app/ 
COPY --from=build-stage /function/target/*.jar /function/app/ 
CMD ["com.example.fn.LabelImageFunction::classify"]

Notice the additional customization that it incorporates, in addition to the default steps like Maven build:

• Automates TensorFlow setup (per the instructions), extracts the TensorFlow Java SDK and the native JNI (.so) libraries
• (as part of the second stage of the Docker build) Copies the JNI libraries to /function/runtime/lib and the SDK JAR to /function/app so that they are available to the function at runtime

Deploying to Oracle Functions

As mentioned previously, you can use the open source Fn CLI to deploy to Oracle Functions. Ensure that you have the latest version.

curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

​You can also download it directly from https://github.com/fnproject/cli/releases.

Oracle Functions Context

Before using Oracle Functions, you have to configure the Fn Project CLI to connect to your Oracle Cloud Infrastructure tenancy.

When the Fn Project CLI is initially installed, it’s configured for a local development context. To configure the Fn Project CLI to connect to your Oracle Cloud Infrastructure tenancy instead, you have to create a new context. The context information is stored in a .yaml file in the ~/.fn/contexts directory. It specifies Oracle Functions endpoints, the OCID of the compartment to which deployed functions belong, the Oracle Cloud Infrastructure configuration file, and the address of the Docker registry to push images to and pull images from.

This is what a context file looks like:

api-url: https://functions.us-phoenix-1.oraclecloud.com oracle.compartment-id: <OCI_compartment_OCID> 
oracle.profile: <profile_name_in_OCI_config> 
provider: oracle 
registry: <OCI_docker_registry>

Oracle Cloud Infrastructure Configuration

The Oracle Cloud Infrastructure configuration file contains information about user credentials and the tenancy OCID. You can create multiple profiles with different values for these entries. Then, you can define the profile to be used by the CLI by using the oracle.profile attribute.

Here is an example configuration file:

[DEFAULT] 
user=ocid1.user.oc1..exampleuniqueID fingerprint=20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34 key_file=~/.oci/oci_api_key.pem tenancy=ocid1.tenancy.oc1..exampleuniqueID pass_phrase=tops3cr3t region=us-ashburn-1
[ORACLE_FUNCTIONS_USER] 
user=ocid1.user.oc1..exampleuniqueID fingerprint=72:00:22:7f:d3:8b:47:a4:58:05:b8:95:84:31:dd:0e key_file=/.oci/admin_key.pem tenancy=ocid1.tenancy.oc1..exampleuniqueID pass_phrase=s3cr3t region=us-phoenix-1

You can define multiple contexts, each stored in a different context file. Switch to the correct context according to your Functions development environment:

fn use context <context_name>

Create the Application

Start by cloning the contents of the GitHub repository:

git clone https://github.com/abhirockzz/fn-hello-tensorflow

Here is the command required to deploy an application:

fn create app <app_name> --annotation oracle.com/oci/subnetIds='["<subnet_ocid>"]'

• <app_name> is the name of the new application.
• <subnet_ocid> is the OCID of the subnet in which to run your function.

For example:

fn create app fn-tensorflow-app --annotation oracle.com/oci/subnetIds='["ocid1.subnet.oc1.phx.exampleuniqueID","ocid1.subnet.oc1.phx.exampleuniqueID","ocid1.subnet.oc1.phx.exampleuniqueID"]'

Deploy the Function

After you create the application, you can deploy your function with the following command:

fn deploy --app <app_name>

<app_name> is the name of the application in Oracle Functions to which you want to add the function.

If you want to use TensorFlow version 1.12.0 (for Java SDK and corresponding native libraries), use the following command:

fn -v deploy --app fn-tensorflow-app

You can also choose a specific version. Ensure that you specify it in pom.xml file before you build the function. For example, if you want to use version 1.11.0:

<dependency> 
 <groupId>org.tensorflow</groupId>     <artifactId>tensorflow</artifactId> 
<version>1.11.0</version> 
<scope>provided</scope> 
</dependency>

To specify the version during function deployment, you can use — build-arg (build argument) as follows:

fn -v deploy --app fn-tensorflow-app --build-arg TENSORFLOW_VERSION=<version>

For example, if you want to use 1.11.0:

fn -v deploy --app fn-tensorflow-app --build-arg TENSORFLOW_VERSION=1.11.0

When the deployment completes successfully, your function is ready to use. Use the fn ls apps command to list down the applications currently deployed. fn-tensorflow-app should be listed.

Time to Classify Images!

As mentioned earlier, the function can accept an image as input and tell you what it is, along with the percentage accuracy.

You can start by downloading some of the recommended images or use images that you already have on your computer. All you need to do is pass them to the function while invoking it:

cat <path to image> | fn invoke fn-tensorflow-app classify

Ok, let’s try this. Can it detect the sombrero in this image?

cat /Users/abhishek/manwithhat.jpg | fn invoke fn-tensorflow-app classify

“366 • 9 • Gringo” (CC BY-NC-ND 2.0) by Pragmagraphr

Result:

This is a ‘sombrero’ Accuracy — 92%

How about a terrier?

cat /Users/abhishek/terrier.jpg | fn invoke fn-tensorflow-app classify

“Terrier” (CC BY-NC 2.0) by No_Water

Result:

This is a 'West Highland white terrier' Accuracy - 88%

What will you classify? :-)

Summary

We just deployed a simple yet fully functional machine learning application in the cloud! Eager to try this out?

Oracle Functions will be generally available in 2019, but we are currently providing access to selected customers through our Cloud Native Limited Availability Program. To learn more about Oracle Functions or to request access, please register. You can also learn more about the underlying open source technology used in Oracle Functions at FnProject.io.

Originally published at blogs.oracle.com on January 7, 2019.