Python bindings for ggml

Python bindings for the ggml tensor library for machine learning.

⚠️ This project is in a very early state and currently only offers the basic low-level bindings to ggml

Documentation

• Getting Started
• API Reference
• Examples

Installation

Requirements

• Python 3.7+
• C compiler (gcc, clang, msvc, etc)

You can install ggml-python using pip:

pip install ggml-python

This will compile ggml using cmake which requires a c compiler installed on your system. To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific flags through the CMAKE_ARGS environment variable. For example to install ggml-python with cuBLAS support you can run:

CMAKE_ARGS="-DGGML_CUBLAS=ON" pip install ggml-python

Options

Option	Description	Default
GGML_CUBLAS	Enable cuBLAS support	OFF
GGML_CLBLAST	Enable CLBlast support	OFF
GGML_OPENBLAS	Enable OpenBLAS support	OFF
GGML_METAL	Enable Metal support	OFF

Usage

import ggml
import ctypes

# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(
    mem_size=16 * 1024 * 1024,
    mem_buffer=None
)
ctx = ggml.ggml_init(params=params)

# Instantiate tensors
x = ggml.ggml_new_tensor_1d(
    ctx,
    ggml.GGML_TYPE_F32,
    ctypes.c_int64(1),
)
a = ggml.ggml_new_tensor_1d(
    ctx,
    ggml.GGML_TYPE_F32,
    ctypes.c_int64(1),
)
b = ggml.ggml_new_tensor_1d(
    ctx,
    ggml.GGML_TYPE_F32,
    ctypes.c_int64(1),
)

# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)

gf = ggml.ggml_build_forward(f)

# Set the input values
ggml.ggml_set_f32(x, ctypes.c_float(2.0))
ggml.ggml_set_f32(a, ctypes.c_float(3.0))
ggml.ggml_set_f32(b, ctypes.c_float(4.0))

# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, ctypes.pointer(gf), 1)

# Get the output value
output = ggml.ggml_get_f32_1d(f, ctypes.c_int(0))
assert output == 16.0

# Free the context
ggml.ggml_free(ctx)

Troubleshooting

If you are having trouble installing ggml-python or activating specific features please try to install it with the --verbose and --no-cache-dir flags to get more information about any issues:

[options] pip install ggml-python --verbose --no-cache-dir --force-reinstall --upgrade

License

This project is licensed under the terms of the MIT license.