SA Model Viewer base

This commit is contained in:
jack
2026-08-29 20:31:31 +01:00
commit 4eab14d4d6
20 changed files with 57452 additions and 0 deletions
+11
View File
@@ -0,0 +1,11 @@
{
"version": "0.0.1",
"configurations": [
{
"name": "modelviewer",
"runtimeExecutable": "node",
"runtimeArgs": ["server.js"],
"port": 3000
}
]
}
+20
View File
@@ -0,0 +1,20 @@
# Copy to /etc/modelviewer.env on the server (chmod 600, owned by root).
PORT=3000
# --- Model source -----------------------------------------------------------
# Set S3_BUCKET to index a bucket. Leave it unset to use the local models/
# folder, which is what development does.
S3_BUCKET=your-bucket
S3_PREFIX=skins/
AWS_REGION=eu-west-2
# --- Where the browser fetches model files from ------------------------------
# Path A: leave unset. Files are requested from /models on this origin and
# nginx proxies that to S3/CloudFront. No CORS needed.
# Path B: set to a CDN origin so the browser fetches S3 directly. Requires a
# CORS rule on the bucket.
# MODELS_BASE_URL=https://cdn.southwest-roleplay.dev/skins
# How often to re-list the bucket, in ms. Default 15 minutes.
# INDEX_REFRESH_MS=900000
+6
View File
@@ -0,0 +1,6 @@
node_modules/
.env
# Extracted game assets - keep them out of the repo
models/*.dff
models/*.txd
+124
View File
@@ -0,0 +1,124 @@
# GTA:SA model viewer
Drop a `.dff` and `.txd` into `models/`, open `/samp/skin/<id>`, and it renders.
The RenderWare files are parsed in the browser — no conversion step, no Blender,
no build tooling.
`/samp/skin/<id>` is the only thing the server exposes. The root, `/index.html`
and any other path return 404.
## Running
```bash
npm install
npm start
```
Then open <http://localhost:3000/samp/skin/1>.
`PORT` and `MODEL_ROOT` are read from the environment, so you can point it at a
folder outside the project:
```bash
MODEL_ROOT=C:/samp/skins npm start
```
See [.env.example](.env.example) for the full set, including the S3 options.
## Getting the default SA-MP skins
The 312 SA-MP skin ids are just GTA:SA ped model ids, so they come out of your
own install rather than off the web:
```bash
node tools/extract-peds.js --game "D:/SteamLibrary/steamapps/common/Grand Theft Auto San Andreas"
```
It reads `data/peds.ide` for the id -> model mapping, pulls each `.dff` and
`.txd` out of `models/gta3.img` and `models/player.img`, and writes them to
`models/<id>.dff` / `models/<id>.txd`. `--game` is optional if the install is
in one of the usual places.
That yields **265 skins**. Three groups are absent for real reasons:
- The gaps (1-6, 8, 42, 65, 74, 86, 119, 149, 208, 265-273, 289+) are the ids
SA-MP treats as invalid. They are simply not in peds.ide.
- Ids 290-299 (`special01`-`special10`) are empty placeholder slots that servers
fill with `AddCharModel`, so there is nothing to extract.
- Id 0 (CJ) has no single model. `player.dff` is an 8KB stub and the game
assembles him from the clothing components in player.img at runtime, so the
extractor skips him rather than writing a placeholder.
Extracted models are gitignored; they're Rockstar's assets, not yours to ship.
## Naming
Files are named by id: `<id>.dff` and `<id>.txd`, in `models/` locally or under
the bucket prefix on S3. The `.txd` is optional; without it the model renders
untextured, and a `.txd` with no matching `.dff` is ignored.
You can drag a `.dff` and `.txd` straight onto the page to preview them without
filing them first, which is handy for checking a custom skin.
## Deployment
The server reads its model list from a local folder by default, or from S3 when
`S3_BUCKET` is set. Either way the list is built once at boot and refreshed on a
timer (`INDEX_REFRESH_MS`, default 15 minutes), so uploading a new skin does not
need a restart.
`MODELS_BASE_URL` controls where the browser fetches model files from:
- **unset** — files are requested from `/models` on this origin, and nginx
proxies that to S3 or CloudFront. Same-origin, so no bucket CORS rule.
- **set to a CDN origin** — the browser fetches S3 directly and the server stays
out of the data path. Needs the CORS rule in
[deploy/bucket-cors.json](deploy/bucket-cors.json).
Ready-to-edit configs live in [deploy/](deploy/): an nginx site that exposes only
`/samp/skin/`, a hardened systemd unit, the bucket CORS rule and a read-only IAM
policy.
If you would rather not put AWS credentials on the box at all, drop the
`@aws-sdk/client-s3` dependency and have `/api/skin/:id` return the URLs without
checking the index. You lose clean 404s for unknown ids — the viewer reports the
failed fetch instead.
## API
- `GET /api/skin/:id` — the URLs for one id, or 404
## What's implemented
`public/js/rw/` is a standalone RenderWare reader with no dependencies:
- `stream.js` — chunk walker. Every RW chunk is a 12 byte header
(`type`, `size`, `libraryID`) followed by its body; containers just hold more
chunks. Both formats below are built on this.
- `dff.js` — clump, frame hierarchy, geometry (positions, normals, UVs, prelit
colours, triangles), material list and texture names.
- `txd.js` — texture dictionary, D3D8 and D3D9 rasters.
- `dxt.js` — DXT1/3/5 decoder, everything ends up as RGBA8.
Verified against 179 dffs and 127 txds (1383 textures) from a SA-MP cache: all
parsed, with every triangle index and material id in range.
Things deliberately left out, in rough order of how likely they are to matter:
- **Animation.** Ped dffs store vertices in bind pose, so they render standing
still, correctly. Posing them would need the Skin PLG (`0x116`) and HAnim PLG
(`0x11E`) chunks turned into a `THREE.Skeleton` plus an IFP reader.
Bind pose has one non-obvious rule: a *skinned* geometry's vertices live in
the space of its atomic frame's **parent**, not the atomic frame itself, so
that one frame is skipped when placing the mesh. The atomic hangs off `Pelvis`
in most peds but off `Root` in a handful, which is why neither "always apply
the frame" nor "always use identity" works. Checked against all 265 extracted
peds: every one stands upright at human height under this rule.
- **Native geometry.** PS2/Xbox dffs use a different vertex layout and are
rejected with a message rather than parsed. PC files are fine.
- **Bin Mesh PLG.** SA stores per-material tristrips in `0x50E`, but the plain
triangle list is always present too, so the viewer groups that by material
instead.
- Environment/specular maps, UV animation, vehicle paint colours, IMG archives.
+8
View File
@@ -0,0 +1,8 @@
[
{
"AllowedOrigins": ["https://model.southwest-roleplay.dev"],
"AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["*"],
"MaxAgeSeconds": 3600
}
]
+16
View File
@@ -0,0 +1,16 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::your-bucket/skins/*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::your-bucket",
"Condition": { "StringLike": { "s3:prefix": "skins/*" } }
}
]
}
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=GTA:SA model viewer
After=network.target
[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/modelviewer
EnvironmentFile=/etc/modelviewer.env
ExecStart=/usr/bin/node server.js
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
[Install]
WantedBy=multi-user.target
+49
View File
@@ -0,0 +1,49 @@
# /etc/nginx/sites-available/model.southwest-roleplay.dev
#
# Only /samp/skin/<id> and the assets that page needs are exposed. Everything
# else, including the root, returns 404. certbot rewrites this for 443 and
# leaves the location blocks untouched.
server {
listen 80;
server_name model.southwest-roleplay.dev;
location /samp/skin/ {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /api/skin/ {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /js/ {
proxy_pass http://127.0.0.1:3000;
}
location /vendor/ {
proxy_pass http://127.0.0.1:3000;
add_header Cache-Control "public, max-age=31536000, immutable";
}
# Path A only: model files come from S3 through nginx, so they stay
# same-origin and no bucket CORS rule is needed. Delete this block if you
# set MODELS_BASE_URL and let the browser fetch the CDN directly.
location /models/ {
proxy_pass https://your-bucket.s3.eu-west-2.amazonaws.com/skins/;
proxy_set_header Host your-bucket.s3.eu-west-2.amazonaws.com;
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-server-side-encryption;
add_header Cache-Control "public, max-age=31536000, immutable";
}
location / {
return 404;
}
}
+1243
View File
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
{
"name": "modelviewer",
"version": "1.0.0",
"description": "GTA:SA DFF/TXD model viewer",
"main": "server.js",
"scripts": {
"start": "node server.js"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.1121.0",
"dotenv": "^17.4.2",
"express": "^4.19.2"
}
}
+93
View File
@@ -0,0 +1,93 @@
import { parseDFF } from './rw/dff.js';
import { parseTXD } from './rw/txd.js';
import { Viewer } from './viewer.js';
const canvas = document.getElementById('view');
const statusEl = document.getElementById('status');
const viewer = new Viewer(canvas);
// The page shows nothing but the model, so a failure is the only thing worth
// putting on screen. Everything else goes to the console.
function setError(text) {
statusEl.textContent = text || '';
statusEl.style.display = text ? 'block' : 'none';
}
async function fetchBuffer(url) {
const res = await fetch(url);
if (!res.ok) throw new Error(`${res.status} fetching ${url}`);
return res.arrayBuffer();
}
function render(name, dffBuffer, txdBuffer) {
const dff = parseDFF(dffBuffer);
const txd = txdBuffer ? parseTXD(txdBuffer) : null;
const report = viewer.load(dff, txd);
console.log(
`${name}: rw ${dff.versionText}, ${report.meshes} mesh(es), ` +
`${report.vertices} verts, ${report.triangles} tris, ` +
`${txd ? txd.textures.size : 0} textures`
);
const warnings = [...dff.errors, ...(txd ? txd.errors : [])];
if (report.missingTextures.size) {
warnings.push('not in txd: ' + [...report.missingTextures].join(', '));
}
warnings.forEach((warning) => console.warn(warning));
setError('');
}
async function loadSkin(id) {
try {
const res = await fetch(`/api/skin/${encodeURIComponent(id)}`);
const meta = await res.json();
if (!res.ok) {
setError(meta.error || `${res.status} loading ${id}`);
return;
}
const [dffBuffer, txdBuffer] = await Promise.all([
fetchBuffer(meta.dff),
meta.txd ? fetchBuffer(meta.txd) : Promise.resolve(null),
]);
render(id, dffBuffer, txdBuffer);
} catch (err) {
console.error(err);
setError(err.message);
}
}
function setupDragAndDrop() {
document.body.addEventListener('dragover', (e) => {
e.preventDefault();
document.body.classList.add('dragging');
});
document.body.addEventListener('dragleave', () => document.body.classList.remove('dragging'));
document.body.addEventListener('drop', async (e) => {
e.preventDefault();
document.body.classList.remove('dragging');
const files = [...e.dataTransfer.files];
const dffFile = files.find((f) => f.name.toLowerCase().endsWith('.dff'));
const txdFile = files.find((f) => f.name.toLowerCase().endsWith('.txd'));
if (!dffFile) {
setError('drop a .dff (and its .txd) together');
return;
}
try {
render(
dffFile.name.replace(/\.dff$/i, ''),
await dffFile.arrayBuffer(),
txdFile ? await txdFile.arrayBuffer() : null
);
} catch (err) {
console.error(err);
setError(err.message);
}
});
}
setupDragAndDrop();
const match = location.pathname.match(/^\/samp\/skin\/(.+)$/);
if (match) loadSkin(decodeURIComponent(match[1]));
+243
View File
@@ -0,0 +1,243 @@
// DFF (RenderWare Clump) reader.
//
// Layout: Clump -> FrameList -> GeometryList -> Geometry* -> MaterialList
// -> Atomic* (pairs a geometry with a frame)
//
// Only the parts a viewer needs are read: transforms, vertex data, triangles
// and material/texture names. Skinning, collision and 2dfx are skipped.
import { RW, Chunk, Reader, versionString } from './stream.js';
const FLAG_TRISTRIP = 0x0001;
const FLAG_TEXTURED = 0x0004;
const FLAG_PRELIT = 0x0008;
const FLAG_NORMALS = 0x0010;
const FLAG_TEXTURED2 = 0x0080;
const FLAG_NATIVE = 0x01000000;
function parseFrameList(chunk) {
const r = chunk.structReader();
const count = r.i32();
const frames = [];
for (let i = 0; i < count; i++) {
// 3x3 basis (right, up, at) then position - already column major for three.js.
const m = r.f32Array(12);
const parent = r.i32();
r.u32(); // matrix flags, unused
frames.push({
parent,
name: '',
matrix: [m[0], m[1], m[2], 0, m[3], m[4], m[5], 0, m[6], m[7], m[8], 0, m[9], m[10], m[11], 1],
});
}
// Each frame's extension follows the struct, in the same order.
let i = 0;
for (const ext of chunk.findAll(RW.EXTENSION)) {
if (i >= frames.length) break;
const nameChunk = ext.find(RW.FRAME_NAME);
if (nameChunk) frames[i].name = nameChunk.reader().fixedString(nameChunk.size);
i++;
}
return frames;
}
function parseTexture(chunk) {
const r = chunk.structReader();
r.u16(); // filter mode
r.u16(); // uv addressing
const strings = chunk.findAll(RW.STRING);
const read = (c) => (c ? c.reader().fixedString(c.size) : '');
return { name: read(strings[0]), mask: read(strings[1]) };
}
function parseMaterial(chunk) {
const r = chunk.structReader();
r.i32(); // flags
const color = [r.u8() / 255, r.u8() / 255, r.u8() / 255, r.u8() / 255];
r.i32(); // unused
const isTextured = r.i32();
let ambient = 1, specular = 1, diffuse = 1;
if (chunk.version > 0x30400) {
ambient = r.f32();
specular = r.f32();
diffuse = r.f32();
}
const texChunk = chunk.find(RW.TEXTURE);
const texture = texChunk ? parseTexture(texChunk) : null;
return { color, isTextured: !!isTextured, ambient, specular, diffuse, texture };
}
function parseMaterialList(chunk) {
const r = chunk.structReader();
const count = r.i32();
const refs = [];
for (let i = 0; i < count; i++) refs.push(r.i32());
const chunks = chunk.findAll(RW.MATERIAL);
const materials = [];
let next = 0;
for (let i = 0; i < count; i++) {
// -1 means "a new material follows", anything else reuses an earlier one.
if (refs[i] >= 0 && materials[refs[i]]) materials.push(materials[refs[i]]);
else if (next < chunks.length) materials.push(parseMaterial(chunks[next++]));
else materials.push(parseMaterial(chunks[chunks.length - 1]));
}
return materials;
}
function parseGeometry(chunk) {
const r = chunk.structReader();
const format = r.u32();
const numTriangles = r.i32();
const numVertices = r.i32();
const numMorphTargets = r.i32();
if (chunk.version < 0x34000) {
r.f32(); r.f32(); r.f32(); // surface properties moved into the material later
}
if (format & FLAG_NATIVE) {
throw new Error('native (PS2/Xbox) geometry is not supported - use a PC dff');
}
let uvSets = (format & 0x00ff0000) >> 16;
if (!uvSets) uvSets = format & FLAG_TEXTURED2 ? 2 : format & FLAG_TEXTURED ? 1 : 0;
let colors = null;
if (format & FLAG_PRELIT) {
colors = new Float32Array(numVertices * 3);
for (let i = 0; i < numVertices; i++) {
colors[i * 3] = r.u8() / 255;
colors[i * 3 + 1] = r.u8() / 255;
colors[i * 3 + 2] = r.u8() / 255;
r.u8(); // prelit alpha, unused
}
}
let uvs = null;
for (let set = 0; set < uvSets; set++) {
const data = r.f32Array(numVertices * 2);
if (set === 0) uvs = data;
}
// Stored as (b, a, materialId, c) - the second vertex comes first and the
// material index sits between the vertices.
const tris = new Uint16Array(numTriangles * 3);
const triMaterial = new Uint16Array(numTriangles);
for (let i = 0; i < numTriangles; i++) {
const b = r.u16();
const a = r.u16();
triMaterial[i] = r.u16();
const c = r.u16();
tris[i * 3] = a;
tris[i * 3 + 1] = b;
tris[i * 3 + 2] = c;
}
let positions = null;
let normals = null;
for (let i = 0; i < numMorphTargets; i++) {
r.f32Array(4); // bounding sphere
const hasVertices = r.u32();
const hasNormals = r.u32();
const verts = hasVertices ? r.f32Array(numVertices * 3) : null;
const norms = hasNormals ? r.f32Array(numVertices * 3) : null;
if (i === 0) {
positions = verts;
normals = norms;
}
}
if (!positions) throw new Error('geometry has no vertex positions');
const matList = chunk.find(RW.MATLIST);
const materials = matList ? parseMaterialList(matList) : [];
// A skinned geometry stores its vertices in bind pose rather than in its
// frame's local space, so the viewer must not apply the frame transform.
const extension = chunk.find(RW.EXTENSION);
const skinned = !!(extension && extension.find(RW.SKIN));
// One draw group per material, so each group can get its own texture.
const order = [...Array(numTriangles).keys()].sort((x, y) => triMaterial[x] - triMaterial[y]);
const indices = new Uint32Array(numTriangles * 3);
const groups = [];
let current = null;
order.forEach((tri, slot) => {
indices[slot * 3] = tris[tri * 3];
indices[slot * 3 + 1] = tris[tri * 3 + 1];
indices[slot * 3 + 2] = tris[tri * 3 + 2];
const material = triMaterial[tri];
if (!current || current.material !== material) {
current = { start: slot * 3, count: 0, material };
groups.push(current);
}
current.count += 3;
});
return {
numVertices,
positions,
normals,
uvs,
colors,
indices,
groups,
materials,
skinned,
tristrip: !!(format & FLAG_TRISTRIP),
};
}
export function parseDFF(buffer) {
const view = new DataView(buffer);
let clump = new Chunk(view, 0);
if (clump.type !== RW.CLUMP) {
// Some files carry a UV animation dictionary or similar before the clump.
clump = null;
for (let offset = 0; offset + 12 <= view.byteLength; ) {
const c = new Chunk(view, offset);
if (c.type === RW.CLUMP) { clump = c; break; }
if (c.size <= 0) break;
offset += 12 + c.size;
}
if (!clump) throw new Error('no clump chunk found - this is not a dff');
}
const r = clump.structReader();
const numAtomics = r.i32();
if (clump.version > 0x33000) {
r.i32(); // lights
r.i32(); // cameras
}
const frameList = clump.find(RW.FRAMELIST);
const frames = frameList ? parseFrameList(frameList) : [];
const geometryList = clump.find(RW.GEOMETRYLIST);
const geometries = [];
const errors = [];
if (geometryList) {
geometryList.findAll(RW.GEOMETRY).forEach((geo, i) => {
try {
geometries.push(parseGeometry(geo));
} catch (err) {
errors.push(`geometry ${i}: ${err.message}`);
geometries.push(null);
}
});
}
const atomics = clump.findAll(RW.ATOMIC).map((atomic) => {
const a = atomic.structReader();
return { frame: a.i32(), geometry: a.i32(), flags: a.u32() };
});
return {
version: clump.version,
versionText: versionString(clump.version),
numAtomics,
frames,
geometries,
atomics,
errors,
};
}
+96
View File
@@ -0,0 +1,96 @@
// Minimal DXT1/3/5 decoder. Everything is decoded to RGBA8 so the viewer has a
// single upload path and doesn't depend on the S3TC extension being present.
function color565(c, out, i) {
out[i] = ((c >> 11) & 0x1f) * 255 / 31;
out[i + 1] = ((c >> 5) & 0x3f) * 255 / 63;
out[i + 2] = (c & 0x1f) * 255 / 31;
}
function buildPalette(data, offset, dxt1) {
const c0 = data[offset] | (data[offset + 1] << 8);
const c1 = data[offset + 2] | (data[offset + 3] << 8);
const p = new Uint8Array(16);
color565(c0, p, 0);
color565(c1, p, 4);
p[3] = p[7] = 255;
if (c0 > c1 || !dxt1) {
for (let i = 0; i < 3; i++) {
p[8 + i] = (2 * p[i] + p[4 + i]) / 3;
p[12 + i] = (p[i] + 2 * p[4 + i]) / 3;
}
p[11] = p[15] = 255;
} else {
for (let i = 0; i < 3; i++) {
p[8 + i] = (p[i] + p[4 + i]) / 2;
p[12 + i] = 0;
}
p[11] = 255;
p[15] = 0; // 1 bit punch-through alpha
}
return p;
}
export function decodeDXT(data, width, height, format) {
const out = new Uint8Array(width * height * 4);
const blocksX = Math.max(1, (width + 3) >> 2);
const blocksY = Math.max(1, (height + 3) >> 2);
const blockSize = format === 1 ? 8 : 16;
let offset = 0;
for (let by = 0; by < blocksY; by++) {
for (let bx = 0; bx < blocksX; bx++, offset += blockSize) {
if (offset + blockSize > data.length) return out;
const colorOffset = format === 1 ? offset : offset + 8;
const palette = buildPalette(data, colorOffset, format === 1);
const bits =
data[colorOffset + 4] |
(data[colorOffset + 5] << 8) |
(data[colorOffset + 6] << 16) |
(data[colorOffset + 7] << 24);
// DXT5 alpha: two endpoints plus 3 bit indices.
let a0 = 0, a1 = 0, alphaBits0 = 0, alphaBits1 = 0;
if (format === 5) {
a0 = data[offset];
a1 = data[offset + 1];
alphaBits0 = data[offset + 2] | (data[offset + 3] << 8) | (data[offset + 4] << 16);
alphaBits1 = data[offset + 5] | (data[offset + 6] << 8) | (data[offset + 7] << 16);
}
for (let py = 0; py < 4; py++) {
for (let px = 0; px < 4; px++) {
const x = bx * 4 + px;
const y = by * 4 + py;
if (x >= width || y >= height) continue;
const pixel = py * 4 + px;
const idx = ((bits >>> (pixel * 2)) & 3) * 4;
const dst = (y * width + x) * 4;
out[dst] = palette[idx];
out[dst + 1] = palette[idx + 1];
out[dst + 2] = palette[idx + 2];
if (format === 1) {
out[dst + 3] = palette[idx + 3];
} else if (format === 3) {
// 4 bits of alpha per pixel, straight from the first 8 bytes.
const nibble = data[offset + (pixel >> 1)];
out[dst + 3] = (pixel & 1 ? nibble >> 4 : nibble & 0x0f) * 17;
} else {
const shift = (pixel % 8) * 3;
const code = ((pixel < 8 ? alphaBits0 : alphaBits1) >>> shift) & 7;
let a;
if (code === 0) a = a0;
else if (code === 1) a = a1;
else if (a0 > a1) a = ((8 - code) * a0 + (code - 1) * a1) / 7;
else if (code === 6) a = 0;
else if (code === 7) a = 255;
else a = ((6 - code) * a0 + (code - 1) * a1) / 5;
out[dst + 3] = a;
}
}
}
}
}
return out;
}
+154
View File
@@ -0,0 +1,154 @@
// RenderWare binary stream: every chunk is a 12 byte header followed by its
// body, and container chunks simply hold more chunks. That one rule covers
// both DFF and TXD.
export const RW = {
STRUCT: 0x01,
STRING: 0x02,
EXTENSION: 0x03,
TEXTURE: 0x06,
MATERIAL: 0x07,
MATLIST: 0x08,
FRAMELIST: 0x0e,
GEOMETRY: 0x0f,
CLUMP: 0x10,
ATOMIC: 0x14,
TEXNATIVE: 0x15,
TEXDICT: 0x16,
GEOMETRYLIST: 0x1a,
BINMESH: 0x50e,
SKIN: 0x116,
HANIM: 0x11e,
FRAME_NAME: 0x253f2fe,
};
// GTA:SA ships RenderWare 3.6.0.3, which packs to 0x1803FFFF.
export const SA_VERSION = 0x36003;
export function unpackVersion(libraryId) {
if (libraryId & 0xffff0000) {
return (((libraryId >> 14) & 0x3ff00) + 0x30000) | ((libraryId >> 16) & 0x3f);
}
return libraryId << 8;
}
export function versionString(version) {
// major.minor.revision.build - the build number lives in the low 6 bits.
return [(version >> 16) & 0xf, (version >> 12) & 0xf, (version >> 8) & 0xf, version & 0x3f].join('.');
}
export class Reader {
constructor(view, start, end) {
this.view = view;
this.pos = start;
this.end = end;
}
get remaining() {
return this.end - this.pos;
}
u8() {
return this.view.getUint8(this.pos++);
}
u16() {
const v = this.view.getUint16(this.pos, true);
this.pos += 2;
return v;
}
u32() {
const v = this.view.getUint32(this.pos, true);
this.pos += 4;
return v;
}
i32() {
const v = this.view.getInt32(this.pos, true);
this.pos += 4;
return v;
}
f32() {
const v = this.view.getFloat32(this.pos, true);
this.pos += 4;
return v;
}
skip(n) {
this.pos += n;
}
bytes(n) {
const v = new Uint8Array(this.view.buffer, this.view.byteOffset + this.pos, n);
this.pos += n;
return v;
}
// Fixed width, null padded name field.
fixedString(n) {
const raw = this.bytes(n);
let len = raw.indexOf(0);
if (len < 0) len = n;
return String.fromCharCode(...raw.subarray(0, len));
}
f32Array(n) {
const out = new Float32Array(n);
for (let i = 0; i < n; i++) out[i] = this.f32();
return out;
}
}
export class Chunk {
constructor(view, offset, limit) {
this.view = view;
this.type = view.getUint32(offset, true);
this.size = view.getUint32(offset + 4, true);
this.version = unpackVersion(view.getUint32(offset + 8, true));
this.start = offset + 12;
this.end = Math.min(this.start + this.size, limit === undefined ? view.byteLength : limit);
}
reader() {
return new Reader(this.view, this.start, this.end);
}
*children() {
let offset = this.start;
// A 12 byte header has to fit before anything can be read.
while (offset + 12 <= this.end) {
const child = new Chunk(this.view, offset, this.end);
yield child;
const step = 12 + child.size;
if (step <= 0) break; // corrupt size, don't spin
offset += step;
}
}
find(type) {
for (const child of this.children()) {
if (child.type === type) return child;
}
return null;
}
findAll(type) {
const out = [];
for (const child of this.children()) {
if (child.type === type) out.push(child);
}
return out;
}
// Convenience for the STRUCT chunk that nearly every container starts with.
structReader() {
const struct = this.find(RW.STRUCT);
if (!struct) throw new Error(`chunk 0x${this.type.toString(16)} has no struct`);
return struct.reader();
}
}
export function readRoot(buffer, expectedType) {
if (buffer.byteLength < 12) throw new Error('file is too small to be a RenderWare stream');
const view = new DataView(buffer);
const root = new Chunk(view, 0);
if (expectedType !== undefined && root.type !== expectedType) {
throw new Error(
`expected chunk 0x${expectedType.toString(16)}, got 0x${root.type.toString(16)} ` +
'(wrong file type, or the file is compressed/encrypted)'
);
}
return root;
}
+179
View File
@@ -0,0 +1,179 @@
// TXD (RenderWare Texture Dictionary) reader.
//
// TextureDictionary -> TextureNative*, each holding a D3D8 or D3D9 raster.
// Everything is converted to RGBA8 at mip level 0.
import { RW, Chunk } from './stream.js';
import { decodeDXT } from './dxt.js';
const FORMAT_MASK = 0x0f00;
const FMT_1555 = 0x0100;
const FMT_565 = 0x0200;
const FMT_4444 = 0x0300;
const FMT_LUM8 = 0x0400;
const FMT_8888 = 0x0500;
const FMT_888 = 0x0600;
const FMT_555 = 0x0700;
const EXT_PAL8 = 0x2000;
const EXT_PAL4 = 0x4000;
function fourCCToDXT(fourCC) {
switch (fourCC) {
case 0x31545844: return 1; // 'DXT1'
case 0x33545844: return 3; // 'DXT3'
case 0x35545844: return 5; // 'DXT5'
default: return 0;
}
}
// D3D rasters are stored BGRA in memory, so red and blue are swapped.
function decodeUncompressed(data, width, height, format, palette, depth) {
const out = new Uint8Array(width * height * 4);
const pixels = width * height;
if (palette) {
for (let i = 0; i < pixels; i++) {
const p = data[i] * 4;
out[i * 4] = palette[p + 2];
out[i * 4 + 1] = palette[p + 1];
out[i * 4 + 2] = palette[p];
out[i * 4 + 3] = palette[p + 3];
}
return out;
}
for (let i = 0; i < pixels; i++) {
const dst = i * 4;
switch (format) {
case FMT_8888: {
const s = i * 4;
out[dst] = data[s + 2];
out[dst + 1] = data[s + 1];
out[dst + 2] = data[s];
out[dst + 3] = data[s + 3];
break;
}
case FMT_888: {
const s = i * (depth === 24 ? 3 : 4);
out[dst] = data[s + 2];
out[dst + 1] = data[s + 1];
out[dst + 2] = data[s];
out[dst + 3] = 255;
break;
}
case FMT_565: {
const c = data[i * 2] | (data[i * 2 + 1] << 8);
out[dst] = ((c >> 11) & 0x1f) * 255 / 31;
out[dst + 1] = ((c >> 5) & 0x3f) * 255 / 63;
out[dst + 2] = (c & 0x1f) * 255 / 31;
out[dst + 3] = 255;
break;
}
case FMT_1555:
case FMT_555: {
const c = data[i * 2] | (data[i * 2 + 1] << 8);
out[dst] = ((c >> 10) & 0x1f) * 255 / 31;
out[dst + 1] = ((c >> 5) & 0x1f) * 255 / 31;
out[dst + 2] = (c & 0x1f) * 255 / 31;
out[dst + 3] = format === FMT_555 || c & 0x8000 ? 255 : 0;
break;
}
case FMT_4444: {
const c = data[i * 2] | (data[i * 2 + 1] << 8);
out[dst] = ((c >> 8) & 0x0f) * 17;
out[dst + 1] = ((c >> 4) & 0x0f) * 17;
out[dst + 2] = (c & 0x0f) * 17;
out[dst + 3] = ((c >> 12) & 0x0f) * 17;
break;
}
case FMT_LUM8: {
out[dst] = out[dst + 1] = out[dst + 2] = data[i];
out[dst + 3] = 255;
break;
}
default:
out[dst] = out[dst + 1] = out[dst + 2] = 128;
out[dst + 3] = 255;
}
}
return out;
}
function parseTextureNative(chunk) {
const r = chunk.structReader();
const platform = r.u32();
r.u32(); // filter and uv addressing modes
const name = r.fixedString(32);
const mask = r.fixedString(32);
const rasterFormat = r.u32();
let dxt = 0;
let hasAlpha = false;
if (platform === 9) {
dxt = fourCCToDXT(r.u32());
} else {
hasAlpha = !!r.u32();
}
const width = r.u16();
const height = r.u16();
const depth = r.u8();
const numLevels = r.u8();
r.u8(); // raster type
const flags = r.u8();
if (platform === 9) {
hasAlpha = !!(flags & 1);
if (!(flags & 8)) dxt = 0;
} else {
dxt = flags; // D3D8 stores the compression mode here (0, 1 or 3)
}
const format = rasterFormat & FORMAT_MASK;
let palette = null;
if (rasterFormat & EXT_PAL8) palette = r.bytes(256 * 4);
else if (rasterFormat & EXT_PAL4) palette = r.bytes(32 * 4);
const size = r.u32();
const data = r.bytes(Math.min(size, r.remaining));
let rgba;
if (dxt) rgba = decodeDXT(data, width, height, dxt);
else if (rasterFormat & EXT_PAL4) rgba = null; // 4 bit palettes are not used by SA
else rgba = decodeUncompressed(data, width, height, format, palette, depth);
return {
name,
mask,
width,
height,
depth,
numLevels,
platform,
hasAlpha,
compression: dxt ? `DXT${dxt}` : palette ? 'PAL8' : `0x${format.toString(16)}`,
rgba,
};
}
export function parseTXD(buffer) {
const view = new DataView(buffer);
const dict = new Chunk(view, 0);
if (dict.type !== RW.TEXDICT) {
throw new Error(`not a txd (root chunk is 0x${dict.type.toString(16)})`);
}
const textures = new Map();
const errors = [];
dict.findAll(RW.TEXNATIVE).forEach((chunk, i) => {
try {
const tex = parseTextureNative(chunk);
if (tex.rgba) textures.set(tex.name.toLowerCase(), tex);
else errors.push(`${tex.name}: unsupported format ${tex.compression}`);
} catch (err) {
errors.push(`texture ${i}: ${err.message}`);
}
});
return { textures, errors };
}
+189
View File
@@ -0,0 +1,189 @@
import * as THREE from 'three';
import { OrbitControls } from '/vendor/OrbitControls.js';
const MISSING_COLOR = 0xb0b0b0;
export class Viewer {
constructor(canvas) {
this.renderer = new THREE.WebGLRenderer({ canvas, antialias: true });
this.renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));
this.renderer.outputColorSpace = THREE.SRGBColorSpace;
this.scene = new THREE.Scene();
this.scene.background = new THREE.Color(0x171a1f);
this.camera = new THREE.PerspectiveCamera(45, 1, 0.01, 2000);
this.camera.position.set(1.5, 1.2, 2.5);
this.controls = new OrbitControls(this.camera, canvas);
this.controls.enableDamping = true;
this.scene.add(new THREE.AmbientLight(0xffffff, 1.1));
const key = new THREE.DirectionalLight(0xffffff, 1.6);
key.position.set(3, 6, 4);
this.scene.add(key);
const fill = new THREE.DirectionalLight(0xffffff, 0.7);
fill.position.set(-4, 2, -3);
this.scene.add(fill);
this.model = null;
this.textures = [];
window.addEventListener('resize', () => this.resize());
this.resize();
this.renderer.setAnimationLoop(() => {
this.controls.update();
this.renderer.render(this.scene, this.camera);
});
}
resize() {
const canvas = this.renderer.domElement;
const width = canvas.clientWidth || 1;
const height = canvas.clientHeight || 1;
this.renderer.setSize(width, height, false);
this.camera.aspect = width / height;
this.camera.updateProjectionMatrix();
}
clear() {
if (this.model) {
this.model.traverse((obj) => {
if (obj.geometry) obj.geometry.dispose();
if (obj.material) [].concat(obj.material).forEach((m) => m.dispose());
});
this.scene.remove(this.model);
this.model = null;
}
this.textures.forEach((t) => t.dispose());
this.textures = [];
}
makeTexture(source) {
const texture = new THREE.DataTexture(source.rgba, source.width, source.height, THREE.RGBAFormat);
texture.colorSpace = THREE.SRGBColorSpace;
texture.wrapS = texture.wrapT = THREE.RepeatWrapping;
texture.magFilter = THREE.LinearFilter;
texture.minFilter = THREE.LinearMipmapLinearFilter;
texture.generateMipmaps = true;
texture.anisotropy = this.renderer.capabilities.getMaxAnisotropy();
// RenderWare UVs put (0,0) at the top left, which matches the raw row order.
texture.flipY = false;
texture.needsUpdate = true;
this.textures.push(texture);
return texture;
}
buildMaterial(material, txd, report) {
const params = {
color: new THREE.Color(material.color[0], material.color[1], material.color[2]),
side: THREE.DoubleSide,
shininess: 8,
};
if (material.texture && material.texture.name) {
const key = material.texture.name.toLowerCase();
const source = txd && txd.textures.get(key);
if (source) {
params.map = this.makeTexture(source);
params.color = new THREE.Color(0xffffff);
if (source.hasAlpha) {
params.transparent = true;
params.alphaTest = 0.35;
params.depthWrite = true;
}
} else {
report.missingTextures.add(material.texture.name);
params.color = new THREE.Color(MISSING_COLOR);
}
}
if (material.color[3] < 1) {
params.transparent = true;
params.opacity = material.color[3];
}
return new THREE.MeshPhongMaterial(params);
}
buildGeometry(geo) {
const geometry = new THREE.BufferGeometry();
geometry.setAttribute('position', new THREE.BufferAttribute(geo.positions, 3));
if (geo.uvs) geometry.setAttribute('uv', new THREE.BufferAttribute(geo.uvs, 2));
if (geo.colors) geometry.setAttribute('color', new THREE.BufferAttribute(geo.colors, 3));
geometry.setIndex(new THREE.BufferAttribute(geo.indices, 1));
geo.groups.forEach((g) => geometry.addGroup(g.start, g.count, g.material));
if (geo.normals) geometry.setAttribute('normal', new THREE.BufferAttribute(geo.normals, 3));
else geometry.computeVertexNormals();
return geometry;
}
// Frames form a tree; an atomic is placed by its frame's world transform.
// Skinned geometry is the exception: its vertices are stored in the space of
// the atomic frame's *parent*, so that frame's own transform is skipped.
worldMatrix(frames, index, skipSelf) {
const matrix = new THREE.Matrix4();
const chain = [];
let i = index;
let guard = 0;
while (i >= 0 && i < frames.length && guard++ < frames.length) {
chain.unshift(frames[i]);
i = frames[i].parent;
}
if (skipSelf) chain.pop();
for (const frame of chain) {
matrix.multiply(new THREE.Matrix4().fromArray(frame.matrix));
}
return matrix;
}
load(dff, txd) {
this.clear();
const report = { missingTextures: new Set(), meshes: 0, triangles: 0, vertices: 0 };
const root = new THREE.Object3D();
// RenderWare is Z up; three.js is Y up.
root.rotation.x = -Math.PI / 2;
const atomics = dff.atomics.length
? dff.atomics
: dff.geometries.map((_, i) => ({ geometry: i, frame: -1 }));
for (const atomic of atomics) {
const geo = dff.geometries[atomic.geometry];
if (!geo) continue;
const mesh = new THREE.Mesh(
this.buildGeometry(geo),
geo.materials.length
? geo.materials.map((m) => this.buildMaterial(m, txd, report))
: new THREE.MeshPhongMaterial({ color: MISSING_COLOR, side: THREE.DoubleSide })
);
mesh.applyMatrix4(this.worldMatrix(dff.frames, atomic.frame, geo.skinned));
mesh.name = (dff.frames[atomic.frame] && dff.frames[atomic.frame].name) || 'atomic' + report.meshes;
root.add(mesh);
report.meshes++;
report.vertices += geo.numVertices;
report.triangles += geo.indices.length / 3;
}
this.scene.add(root);
this.model = root;
this.frame();
return report;
}
frame() {
if (!this.model) return;
const box = new THREE.Box3().setFromObject(this.model);
if (box.isEmpty()) return;
const size = box.getSize(new THREE.Vector3());
const center = box.getCenter(new THREE.Vector3());
const radius = Math.max(size.x, size.y, size.z) * 0.5 || 1;
this.controls.target.copy(center);
this.camera.position.copy(center).add(new THREE.Vector3(radius * 1.4, radius * 0.8, radius * 2.2));
this.camera.near = radius / 100;
this.camera.far = radius * 100;
this.camera.updateProjectionMatrix();
this.controls.update();
}
}
+1506
View File
File diff suppressed because it is too large Load Diff
+53316
View File
File diff suppressed because one or more lines are too long
+40
View File
@@ -0,0 +1,40 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>GTA:SA model viewer</title>
<style>
:root { color-scheme: dark; }
* { box-sizing: border-box; }
body {
margin: 0; height: 100vh; overflow: hidden;
background: #171a1f; color: #e6e9ef;
font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
#view { display: block; width: 100vw; height: 100vh; }
body.dragging::after {
content: 'drop .dff + .txd';
position: fixed; inset: 12px; border: 2px dashed #5b9dd9; border-radius: 8px;
display: grid; place-items: center; font-size: 18px; color: #5b9dd9;
background: rgba(23, 26, 31, .7); pointer-events: none;
}
/* Only ever shown when something failed to load. */
#status {
position: fixed; top: 50%; left: 50%; transform: translate(-50%, -50%);
padding: 10px 14px; border-radius: 6px; text-align: center;
background: rgba(20, 23, 28, .88); border: 1px solid #5a3030; color: #e06c6c;
}
</style>
<script type="importmap">
{ "imports": { "three": "/vendor/three.module.js" } }
</script>
</head>
<body>
<canvas id="view"></canvas>
<div id="status" style="display: none"></div>
<script type="module" src="/js/main.js"></script>
</body>
</html>
+126
View File
@@ -0,0 +1,126 @@
require('dotenv').config();
const express = require('express');
const path = require('path');
const fs = require('fs');
const app = express();
const PORT = process.env.PORT || 3000;
// Where the model files actually live. Set S3_BUCKET to index a bucket;
// otherwise the local models/ folder is used, which is what development does.
const S3_BUCKET = process.env.S3_BUCKET || '';
const S3_PREFIX = normalisePrefix(process.env.S3_PREFIX || 'skins/');
const MODEL_ROOT = process.env.MODEL_ROOT || path.join(__dirname, 'models');
// Where the browser fetches model files from. The default keeps them on this
// origin under /models, which nginx can proxy to S3. Point it at a CDN origin
// instead to have the browser fetch them directly.
const MODELS_BASE_URL = (process.env.MODELS_BASE_URL || '/models').replace(/\/+$/, '');
const INDEX_REFRESH_MS = Number(process.env.INDEX_REFRESH_MS || 15 * 60 * 1000);
function normalisePrefix(prefix) {
const trimmed = prefix.replace(/^\/+|\/+$/g, '');
return trimmed ? trimmed + '/' : '';
}
// id -> { dff: true, txd: true }. Built once at boot and refreshed on a timer
// so newly uploaded skins appear without a restart.
let index = new Map();
let indexError = null;
function buildIndex(keys, prefix) {
const out = new Map();
for (const key of keys) {
if (!key.startsWith(prefix)) continue;
const name = key.slice(prefix.length);
const match = /^([^/]+)\.(dff|txd)$/i.exec(name);
if (!match) continue;
const entry = out.get(match[1]) || {};
entry[match[2].toLowerCase()] = true;
out.set(match[1], entry);
}
// A txd on its own is not a model.
for (const [id, entry] of out) if (!entry.dff) out.delete(id);
return out;
}
async function listBucketKeys() {
// Required lazily so local development needs no AWS dependency installed.
const { S3Client, ListObjectsV2Command } = require('@aws-sdk/client-s3');
const client = new S3Client({ region: process.env.AWS_REGION });
const keys = [];
let token;
do {
const page = await client.send(
new ListObjectsV2Command({ Bucket: S3_BUCKET, Prefix: S3_PREFIX, ContinuationToken: token })
);
for (const object of page.Contents || []) keys.push(object.Key);
token = page.IsTruncated ? page.NextContinuationToken : undefined;
} while (token);
return keys;
}
function listLocalKeys() {
if (!fs.existsSync(MODEL_ROOT)) return [];
return fs.readdirSync(MODEL_ROOT, { withFileTypes: true })
.filter((entry) => entry.isFile())
.map((entry) => entry.name);
}
async function refreshIndex() {
try {
index = S3_BUCKET
? buildIndex(await listBucketKeys(), S3_PREFIX)
: buildIndex(listLocalKeys(), '');
indexError = null;
} catch (err) {
// Keep serving the previous index rather than going dark on a transient
// S3 error; the next refresh will pick it up.
indexError = err.message;
console.error('model index refresh failed:', err.message);
}
}
app.use(express.static(path.join(__dirname, 'public'), { index: false }));
// In production the model files come from S3 via nginx or a CDN, so this only
// serves them during local development.
if (!S3_BUCKET) {
app.use('/models', express.static(MODEL_ROOT, { immutable: true, maxAge: '365d' }));
}
app.get('/api/skin/:id', (req, res) => {
const entry = index.get(req.params.id);
if (!entry) {
if (!index.size && indexError) {
return res.status(503).json({ error: 'Model index unavailable' });
}
return res.status(404).json({ error: `No model for "${req.params.id}"` });
}
res.json({
id: req.params.id,
name: req.params.id,
dff: `${MODELS_BASE_URL}/${req.params.id}.dff`,
txd: entry.txd ? `${MODELS_BASE_URL}/${req.params.id}.txd` : null,
});
});
app.get('/samp/skin/:id', (req, res) => {
res.sendFile(path.join(__dirname, 'public', 'viewer.html'));
});
// Nothing else exists - no root page, no catalogue.
app.use((req, res) => res.status(404).type('text/plain').send('Not found'));
refreshIndex().then(() => {
setInterval(refreshIndex, INDEX_REFRESH_MS).unref();
app.listen(PORT, () => {
console.log(`modelviewer -> http://localhost:${PORT}/samp/skin/7`);
console.log(`models from -> ${S3_BUCKET ? `s3://${S3_BUCKET}/${S3_PREFIX}` : MODEL_ROOT}`);
console.log(`served from -> ${MODELS_BASE_URL}`);
console.log(`indexed -> ${index.size} skins`);
});
});