Remove unused build logic, update documentation

Author: alonre24

docs/developer-backends.md

@@ -1,14 +1,17 @@
 # RedisAI Development Backends
 
-This document describes how a backend for RedisAI can be built, from this repository. It highlights the supported compilation devices on a per-backend basis, and highlights the tools and commands required.  Unless indicated otherwise, a backend is compiled in a docker, which is responsible for the configuration and installation of all tools required for a given backend on a per-platform basis.
+This document describes how ONNXRuntime backend can be built from this repository.
+We build ONNXRuntime library with DISABLE_EXTERNAL_INITIALIZERS=ON build flag, which is not the default configuration. This means that loading ONNX models that use external files to store the initial (usually very large) values of the model's operations, is invalid. That is, initializers values must be part of the serialized model, which is also the standard use case.
+
+It is compiled in a docker, which is responsible for the configuration and installation of all tools required the build process.
 
 To follow these instructions, this repository must be cloned with all of its submodules (i.e *git clone --recursive https://github.com/redisai/redisai*)
 
-GNU Make is used as a runner for the dockerfile generator. Python is the language used for the generator script, and jinja is the templating library used to create the docker file from a template *dockerfile.tmpl* that can be found in the directory of a given backend listed below.
+GNU Make is used as a runner for the dockerfile generator. Python is the language used for the generator script, and jinja is the templating library used to create the docker file from a template *dockerfile.tmpl* that can be found in the `/opt/build/onnxruntime` directory.
 
-## Tools
+### Tools
 
-Building the backends requires installation of the following tools:
+Building the backend requires installation of the following tools:
 
 1. gnu make
 1. python (3.0 or higher)
@@ -26,12 +29,6 @@ pip install jinja
 
 -------
 
-## Backends
-
-### onnxruntime
-
-We build Onnxruntime library with DISABLE_EXTERNAL_INITIALIZERS=ON build flag. This means that loading ONNX models that use external files to store the initial (usually very large) values of the model's operations, is invalid. That is, initializers values must be part of the serialized model, which is also the standard use case.
-
 **Compilation target devices:**
 
 1. x86\_64 bit linux systems

docs/developer.md

@@ -170,9 +170,16 @@ sudo ./opt/system-setup.py
 
 **Building**
 
-To compile RedisAI, run *make -C opt all*, from the root of the repository.
+Before compiling RedisAI module, make sure that you have the backends' shared libraries in the `deps` directory. These can be easily obtained by running:
+```./get_deps```
+or
+```./get_deps gpu```
+for having GPU support.
+Except for ONNXRuntime backend which is stored in a S3 bucket, other backends' binary files are downloaded directly from their official websites.
+Further details about building ONNXRuntime backend for RedisAI is described in [this document](developer-backends.md).
+
+To compile RedisAI, run `make -C opt` or `make -C opt GPU=1`, from the root of the repository. This will create the module's shared library `redisai.so` in `instali-cpu`/`install-gpu` directory respectively. 
 
-Build the backends is described in [this document](developer-backends.md).
 
 ### Testing
 

get_deps.sh

@@ -29,9 +29,9 @@ if [[ $1 == --help || $1 == help ]]; then
 		VERBOSE=1          Print commands
 		FORCE=1            Download even if present
 		WITH_DLPACK=0      Skip dlpack
-		WITH_TF=0          Skip Tensorflow or download from S3 repo
-		WITH_TFLITE=0      Skip TensorflowLite  or download from S3 repo
-		WITH_PT=0          Skip PyTorch or download from S3 repo
+		WITH_TF=0          Skip Tensorflow or download official version
+		WITH_TFLITE=0      Skip TensorflowLite or download official version
+		WITH_PT=0          Skip PyTorch or download official version
 		WITH_ORT=0         Skip OnnxRuntime or download from S3 repo
 		OS=                Set, to override the platform OS
 		ARCH=0             Set, to override the platform ARCH
@@ -99,17 +99,17 @@ clean_and_fetch() {
     echo "Done."
 }
 
-# This is fot torch backend, which comes in a zip file
+# This is foר torch backend, which comes in a zip file
 clean_and_fetch_torch() {
   archive=$1
   src_url=$2
 
-  [[ $FORCE == 1 ]] && rm -rf ${product}  # FORCE is from the env
-  [[ $FORCE != 1 ]] && [[ -d ${product} ]]  && echo "${product} is in place, skipping. Set FORCE=1 to override. Continuing." && return
-  echo "Installing ${product} from ${src_url} in `pwd`..."
-
-  wget -q -O archive src_url
-
+  [[ $FORCE == 1 ]] && rm -rf libtorch  # FORCE is from the env
+  [[ $FORCE != 1 ]] && [[ -d libtorch ]]  && echo "libtorch is in place, skipping. Set FORCE=1 to override. Continuing." && return
+  echo "Installing libtorch from ${src_url} in `pwd`..."
+  LIBTORCH_ZIP=libtorch-${DEVICE}-${PT_VERSION}.zip
+  wget -q -O ${LIBTORCH_ZIP} ${src_url}
+  unzip -q -o ${LIBTORCH_ZIP}
 }
 
 
@@ -181,14 +181,13 @@ fi # WITH_TFLITE
 ####################################################################################### LIBTORCH
 
 PT_BUILD=cpu
-PT_ARCH=x86_64
 if [[ $OS == linux ]]; then
     PT_OS=linux
     if [[ $GPU == 1 ]]; then
         PT_BUILD=cu111
     fi
     if [[ $ARCH == x64 ]]; then
-        PT_REPACK=1
+        PT_ARCH=x86_64
     else
         echo "Only x64 is supported currently"
     fi
@@ -225,6 +224,7 @@ if [[ $OS == linux ]]; then
     fi
 else
     echo "Only Linux OS is supported currently"
+fi
 
 ORT_ARCHIVE=onnxruntime-${ORT_OS}-${ORT_ARCH}${ORT_BUILD}-${ORT_VERSION}.tgz
 if [[ $WITH_ORT != 0 ]]; then