RAVES preprocessing

Installation and usage

You can either import the main functions into your own Python script, or run the preprocessing from the command line. To import in a Python script, clone the repository and install it with

# From the repository root:
pip install -e .

Then, in your script, use one or more of the following lines:

from raves import raves
from raves import compute_ART
from raves import compute_MoDART

# The '__main__' scope is necessary because `compute_ART` (and, by extension, `raves`)
#  makes use of multiprocessing, unless you manually set `multiprocess_pool_size=1`.
if __name__ == '__main__':
    raves("path/to/your/environment/folder")

To launch from the command line, run

# After installation:
$ raves "path/to/your/environment/folder"

# Alternatively (works without pip-install):
$ python -m raves "path/to/your/environment/folder"

In the latter case (install-free alternative), the command needs to be run from the root directory of the repository.

Examples

The repository includes example input files describing different environments, and example Python scripts illustrating how to use the package. The example scripts are:

• echogram_comparison.py
  • For a set of sound source and listener positions, generate echograms with both TD-ART and MoD-ART, and compare the two approaches.
• residue_heatmaps.py
  • Consider three sound source positions, and a dense grid of listener positions covering the space. Evaluate and display the residues for all listener positions, for the three top modes of the decomposition.
• response_noiseshaping.py
  • Generate audible room impulse responses (for convolution) based on echograms generated by MoD-ART. Listen (WAV)

RAVES workflow

• Prepare the environment description (input files materials.csv, mesh.mtl, mesh.obj) following the file format outlined in the following section. Place all three files into the same folder (the folder's name will be the name of your environment).
• Run the raves script, either from a command line console or from a Python script of your own, as shown above. In either case, the first argument should be the path to your environment folder.
• The files path_indexing.mtx and MoD-ART.csv (among others) will be created in your environment folder. Copy the five files materials.csv, mesh.mtl, mesh.obj, path_indexing.mtx, and MoD-ART.csv into your Unity project assets, matching the folder structure shown below.

Unity project folder structure

Place your mesh files in PythonExports/YourEnvironmentName/ as shown in the following example. The first time your environment is loaded in the Unity editor, a file named YourEnvironmentName.prefab will be automatically generated in ProcessedPrefabs. Make sure the RAVES-Unity GitHub repository is cloned in the Assets folder as shown.

UnityProjectName/
├── Assets/
│   ├── RAVES-Unity/
│   │   └── [cloned GitHub repository]
│   ├── Resources/
│   │   ├── ProcessedPrefabs/
│   │   │   ├── YourEnvironmentName.prefab
│   │   │   ├── YourOtherEnvironmentName.prefab
│   │   │   └── [...]
│   │   ├── PythonExports/
│   │   │   ├── YourEnvironmentName/
│   │   │   │   ├── materials.csv
│   │   │   │   ├── mesh.mtl
│   │   │   │   ├── mesh.obj
│   │   │   │   ├── MoD-ART.csv
│   │   │   │   └── path_indexing.csv
│   │   │   ├── YourOtherEnvironmentName/
│   │   │   │   ├── materials.csv
│   │   │   │   ├── mesh.mtl
│   │   │   │   ├── mesh.obj
│   │   │   │   ├── MoD-ART.csv
│   │   │   │   └── path_indexing.csv
│   │   │   └── [...]
│   │   └── [...]
│   └── [...]
└── [...]

Preparing the environment mesh

The 3D mesh describing the environment needs to be specially prepared for the acoustic radiance transfer (ART) model. In addition to the shape, size, position, and material of each triangle, the input mesh needs to specify the discrete surface patches to be used by ART. The next section provides a few considerations on how to design this surface segmentation — for now, let's consider a basic example.

Let us consider the trivial case of a "shoebox" room (a cuboid). Such a room is formed by six rectangles (floor, ceiling, north, south, east, west). For the sake of making a toy example, we will model each wall as a single surface patch for ART — in reality, this would lead to a very coarse approximation; one should subdivide each wall into smaller patches. Let's say that the floor of our room is carpeted, the ceiling is plaster, and all four side walls are painted concrete. Then, the six patches will be denoted by the following identifiers:

1. Patch_1_Mat_Carpet
2. Patch_2_Mat_Plaster
3. Patch_3_Mat_PaintedConcrete
4. Patch_4_Mat_PaintedConcrete
5. Patch_5_Mat_PaintedConcrete
6. Patch_6_Mat_PaintedConcrete

Note the format of these identifiers: Patch_{i}_Mat_{material}, where i is the unique index of the surface patch for ART, and material is the label of its acoustic material. Multiple patches can feature the same material, but only one material can be assigned to each individual patch.

Later sections in this README give detailed instructions on how to format your input files using these identifiers.

Patch design considerations

The effects of surface patch design on ART simulations are still understudied, but we recommend a few rules-of-thumb:

• each surface patch should form a single polygon (coplanar and connected triangles);
• each surface patch should be relatively compact (avoid long and narrow polygons);
• all surface patches should have roughly the same area.

The quality of the numerical integral in compute_ART is controlled by two variables: the number of surface sample points per square meter, and the number of rays in a hemisphere (i.e., number of direction sample points per $2\pi$ steradian). The functions assess_ART and assess_ART_on_grid are provided as tools to assess the numerical accuracy of integrals for different values of these parameters.

If the computation takes a long time, try using larger patches, or decrease the density of sample points and rays. The former approach will decrease the directional resolution of the ART model, while the latter will decrease the numerical accuracy of the surface integrals.

Modifying the materials of an existing ART model

If you wish to modify the surface material properties after running compute_ART but before running compute_MoDART, it is possible to do so without having to re-run the most expensive ART computations. All you need to do is update your existing input files (read the next section, Input files, if you haven't yet) before re-running the script.

First, modify the contents of the input file materials.csv and, if necessary, the material Mat_{material} assigned to each surface patch in mesh.mtl and mesh.obj. Take care not to change any other part of mesh.mtl and mesh.obj: the mesh geometry and surface patches, including the patch indices, must remain unchanged w.r.t. the original ART computation.

After making your changes to the input files, run the compute_ART script again. It will detect the existing kernel files (which are independent of material properties) and use them alongside the new material data, instead of performing the numerical surface integration again. If you actually want to overwrite the existing numerical integration results, run the script with argument --overwrite.

Input files

The files describing your environment need to be follow the format specifications outlined in the following. The example files shown here can be found in the example environments\Shoebox_6_patches folder; they are related to the trivial room described in the previous section.

mesh.obj

The mesh geometry should be provided in basic Wavefront format (mesh.obj+mesh.mtl). The first line of mesh.obj should be mtllib mesh.mtl, with mesh.mtl being placed in the same folder as mesh.obj. All other lines of mesh.obj should be restricted to vertex definitions v, face definitions f, comments #, and materials assignments usemtl, as detailed in the following. Any other lines will be ignored by the acoustic analysis, but may alter the visual appearance of the mesh in Unity.

Vertex definition lines start with v followed by three floating-point values, separated by spaces. These are the vertex's three-dimensional coordinates. Optionally, the line may include an inline comment (starting with #) specifying the vertex index. Note that vertex indices start from 1. These comments only serve as a human-readable reference, and are ignored by the parser. Avoid using this type of "inline" comment on lines starting with f or usemtl: they create trouble for the Unity parser.

Face definition lines start with f followed by three integer numbers, separated by spaces. These are the indices of the vertices forming each face, listed counter-clockwise around the surface normal. While the Wavefront format supports polygonal faces with more than three vertices, our code only supports triangles. The vertex indices refer to the order in which vertices are defined in the same file.

Comment lines start with # and are ignored by all parsers. You may use these to label groups of surface patches, e.g., # Room 1 floor. Blank lines are also ignored.

Material assignment lines must follow the format usemtl Patch_{i}_Mat_{material}, where i is the ART surface patch index and material is a string identifying the surface material. The patch indices should range from 1 to the number of patches. The material identifier must only contain ASCII letters, digits, or underscores. Each usemtl line applies to all faces defined in following lines, until the next usemtl line. In the example below, each surface patch is a rectangle formed by two adjacent triangles. Our code requires all triangles in each patch to be coplanar.

Example

mtllib mesh.mtl

################################ Vertices

v 0.0 0.0 0.0                  # Vertex 1
v 0.0 0.0 3.0                  # Vertex 2
v 0.0 7.0 0.0                  # Vertex 3
v 0.0 7.0 3.0                  # Vertex 4
v 4.0 0.0 0.0                  # Vertex 5
v 4.0 0.0 3.0                  # Vertex 6
v 4.0 7.0 0.0                  # Vertex 7
v 4.0 7.0 3.0                  # Vertex 8

################################ Faces

# Floor
usemtl Patch_1_Mat_Carpet
f 3 1 5
f 3 5 7
# Ceiling
usemtl Patch_2_Mat_Plaster
f 2 4 6
f 6 4 8
# West
usemtl Patch_3_Mat_PaintedConcrete
f 4 2 1
f 3 4 1
# North
usemtl Patch_4_Mat_PaintedConcrete
f 3 8 4
f 3 7 8
# East
usemtl Patch_5_Mat_PaintedConcrete
f 7 6 8
f 7 5 6
# South
usemtl Patch_6_Mat_PaintedConcrete
f 2 6 5
f 1 2 5

mesh.mtl

The mesh.mtl file should contain a definition for each patch ID string Patch_{i}_Mat_{material} which appears in mesh.obj. Note that these strings must match exactly the ones given in mesh.obj. Each definition consists of two lines: the first one is newmtl Patch_{i}_Mat_{material}, and the second one is Kd <red> <green> <blue> specifying the (diffuse) RGB color of the patch. More lines may be added to specify other visual properties, but the two lines above are the bare minimum for correct parsing.

The material properties defined in mesh.mtl are purely visual, and have no bearing on the acoustic processing. This file only serves to ensure that mesh materials are imported correctly in Unity. Unity disregards the materials mentioned in the .obj if they have no matching definition in the .mtl. Nevertheless, these are the visual properties that will be displayed in the Unity editor, so you may find them useful for visual validation of the patch assignment.

Example

newmtl Patch_1_Mat_Carpet
Kd 1.0 0.7232 0.102
newmtl Patch_2_Mat_Plaster
Kd 0.7154 0.7154 0.7154
newmtl Patch_3_Mat_PaintedConcrete
Kd 0.0 0.5751 0.898
newmtl Patch_4_Mat_PaintedConcrete
Kd 0.0 0.5751 0.898
newmtl Patch_5_Mat_PaintedConcrete
Kd 0.0 0.5751 0.898
newmtl Patch_6_Mat_PaintedConcrete
Kd 0.0 0.5751 0.898

materials.csv

The first line of materials.csv should start with Frequencies and report the center frequencies of the desired bands. These values must form a contiguous range of valid octave bands.
Note: If the first line consists of the word Frequencies and nothing else, all processing will be run in "broadband mode". A frequency of 0Hz is assumed, leading to negligible air absorption. All following lines (absorption/scattering) must report a single value.

Following lines report each material's absorption and scattering coefficients. There must be exactly two lines for each material; the first holds the absorption coefficients, and the second holds the scattering coefficients. Each coefficient line may either present as many floating point values as there are frequency bands, or a single floating point value. In the latter case, the same value is applied in all frequency bands. In the example below, absorption coefficients are specified for each band, whereas scattering coefficients are frequency-independent.

Example

Frequencies, 250.0, 500.0, 1000.0, 2000.0, 4000.0
Carpet, 0.13, 0.6, 0.24, 0.28, 0.32
Carpet, 0.3
Plaster, 0.38, 0.21, 0.15, 0.25, 0.31
Plaster, 0.2
PaintedConcrete, 0.05, 0.06, 0.07, 0.09, 0.08
PaintedConcrete, 0.1

Output files

Scattering matrices

Scattering matrices are stored in Matrix Market format (.mtx), because they are very sparse. Note that these matrices are based on the ART formulation where the quantity traversing propagation paths is power, as opposed to averaged radiance. This aspect is discussed further in ART_theory.md.

The compute_ART script writes scattering matrices in the following files:

• ART_kernel_diffuse.mtx
  • Diffuse (Lambertian) component of the scattering matrix: no energy losses, all scattering coefficients set to 1.
• ART_kernel_specular.mtx
  • Specular component of the scattering matrix: no energy losses, all scattering coefficients set to 0.
• ART_kernel_band_<band_index>.mtx for each frequency band
  • These are the actual scattering matrices, per frequency band. Frequency band indices start from 1. The diffuse and specular components are weighted based on the scattering coefficient of each patch in the specified frequency band, summed together, and then scaled based on the absorption coefficient of each patch in the specified frequency band. Air absorption losses are also included in the scattering matrix, based on propagation path lengths.

Note that the rows and columns of all these matrices follow the propagation path indexing specified in path_indexing.mtx (see below).

Propagation paths

The compute_ART script writes propagation path details in the following files:

• path_indexing.mtx
  • Sparse, square, integer-valued matrix relating each pair of surface patch indices to the index of a propagation path. Zero elements (not reported in the file) denote invalid paths; patch and path indices both start from 1. This is one of the files which should be copied in the Unity asset folder.
• path_delays.csv
  • Propagation path delays, in seconds. Given by the path lengths (see below) divided by the sound speed, which is computed based on the given temperature.
• path_lengths.csv
  • Propagation path lengths, in meters. Defined as the average distance between pairs of points in the double surface integral between the two surface patches at either end of the path.
• path_etendues.csv
  • Propagation path etendues, i.e., product of projected area and solid angle integrated between the two surface patches at either end of the path. This is required to prepare the exported MoD-ART eigenvectors to be used in Unity. This aspect is discussed further in ART_theory.md.

Note that path_lengths.csv and path_etendues.csv only report valid paths, following the indices in path_indexing.mtx.

Example path_indexing.mtx

This example file is truncated for brevity. See the full file at example environments\Shoebox_6_patches\path_indexing.mtx.

%%MatrixMarket matrix coordinate integer general
%Relates each pair of surface patch indices to the index of a propagation path. Zero elements denote invalid paths; patch and path indices both start from 1.
6 6 30
1 2 1
1 3 2
[...]
6 4 29
6 5 30

MoD-ART.csv

The compute_MoDART script writes energy decay mode details in the MoD-ART.csv and MoD-ART extra.csv files. In the former, the number of modes in each frequency band is capped at a given threshold, whereas the latter also includes any additional modes found above the limit. Each mode is characterized by three consecutive lines. The first line of each mode contains only two elements: an integer and a floating point value. The integer value is the index of the frequency band to which this mode pertains (starting from 1, same as ART_kernel_band_<band_index>.mtx). The floating point value is the $T_{60}$ of this mode, in seconds. The second and third lines of each mode contain the right and left eigenvectors, respectively.

The eigenvectors reported here are not defined exactly as in the papers. In preparation for their use at runtime, the right eigenvector is left-multiplied by the scattering matrix, and then divided by the path etendues. Both vectors are multiplied by $4 \pi$. The reasons for these weights are discussed in ART_theory.md.

By default (this can be disabled), the compute_MoDART script also creates two .png files which illustrate the $T_{60}$ values of all discovered energy modes per frequency band. The $T_{60}$ values are plotted in seconds, on either a linear scale or a logarithmic scale (in separate .png files). The filenames also report the sample rate which was used to analize the ART model.

Example MoD-ART.csv

This example file is truncated for brevity. See the full file at example environments\Shoebox_6_patches\MoD-ART.csv.

1, 0.660603515010635
0.014885154368430433, 0.030866274534091905, [...], 0.035148496266458835
0.3066213909592864, 0.31613603861343337, [...], 0.3603800572221311
1, 0.18992986366284817
0.00984441008380384, 0.016160275255061387, [...], 0.019030896267005576
0.034675649884407474, 0.03029173398129135, [...], -0.01772954870081315
[...]
5, 0.07260033445599644
-0.0003530233074883864, -0.0024277787532403397, [...], 0.05078152146757643
-0.0038374243645054023, -0.026565491779966813, [...], 0.34907987912517385