B_BIG storm 700 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Unmodified B_BIG.exe × 700 remote instances produces 70 000 clean TestIndex rows (0 blank fields, 700 distinct INS, CDX IDX01/02/03 intact).

Architecture: Three server/ACE correctness patches — OpenIndex waits on the existing per-path create mutex; SetOrder treats iid 0/missing as natural order and stops mapping nacks to 5000; append takes a blocking record lock (rollback if it cannot). The C++ no-barrier storm is the fast gate; Pritpal’s exe is the product gate.

Tech Stack: OpenADS C++17, ACE ABI, in-process openads::network::Server, doctest, Harbour B_BIG.exe x86 + ace32.dll.

Spec: docs/superpowers/specs/2026-09-02-bbig-storm-700-design.md

Task 1: OpenIndex vs INDEX ON must not return 5103

Files:

  • Modify: src/abi/ace_exports.cpp (AdsOpenIndex local path, ~13330–13500)
  • Modify: src/platform/file_win32.cpp (os_error)
  • Modify: src/platform/file_posix.cpp (errno map, same policy)
  • Test: tests/unit/abi_openindex_create_race_test.cpp (new)

  • Modify: tests/CMakeLists.txt (add the .cpp next to abi_remote_create_stress_test.cpp)

  • Step 1: Write the failing test

New file tests/unit/abi_openindex_create_race_test.cpp. One thread calls AdsCreateIndex61 three tags on a fresh bag; 16 other threads loop AdsOpenIndex on that bag with no barrier. Allowed return codes per OpenIndex attempt: 0, 7040, 5059/5066 (not found). Forbidden: 5103, 5000, 6106 during the create window. After the creator joins, every opener retries until 0. Then AdsGetNumIndexes ≥ 3 and a Skip walk equals the row count (10).

Pattern the server/client setup on tests/unit/abi_remote_create_stress_test.cpp (Server srv; srv.start("127.0.0.1", 0); + tcp://127.0.0.1:<port>/<dir>). Run remote (that’s the B_BIG path).

  • Step 2: Run it — must FAIL
tests\ttest.cmd "*OpenIndex*create*race*"

Expected: FAIL with a 5103 (or 5000) from an opener that hit the bag mid-create.

  • Step 3: Hold create_path_mu_for in AdsOpenIndex

In the local AdsOpenIndex body, after path is resolved and before CdxIndex::list_tags / idx->open:

std::lock_guard<std::mutex> bag_lk(create_path_mu_for(path));

AdsCreateIndex61 already takes this mutex around the create-or-attach decision (ace_exports.cpp ~14272). OpenIndex must wait for that critical section to finish so list_tags never sees a half-written header.

Do not hold state().mu while waiting on create_path_mu_for (comment on the mutex: leaf lock).

Session Opcode::OpenIndex already calls AdsOpenIndex; no duplicate lock in session.cpp.

  • Step 4: Map sharing-violation to 7040

file_win32.cpp os_error:

e.code = (code == ERROR_FILE_NOT_FOUND || code == ERROR_PATH_NOT_FOUND)
             ? 5103
             : (code == ERROR_SHARING_VIOLATION ||
                code == ERROR_LOCK_VIOLATION)
                   ? 7040
                   : 5000;

POSIX sibling: EACCES/ETXTBSY/EAGAIN on open → 7040; ENOENT stays 5103.

  • Step 5: Re-run the race test — PASS

Same ttest.cmd pattern. Also run *create*concurrent* and *remote create/index/append storm* to confirm the old storms stay green.

  • Step 6: Commit
git add src/abi/ace_exports.cpp src/platform/file_win32.cpp src/platform/file_posix.cpp tests/unit/abi_openindex_create_race_test.cpp tests/CMakeLists.txt
git -c user.name="Kimi3" -c user.email="mimo@opencode.ai" commit -m "Fix OpenIndex 5103 during concurrent INDEX ON

Pritpal Bedi B_BIG N=700 logged AE_TABLE_CORRUPTED on TestIndex.cdx
while another session was still in AdsCreateIndex61. Hold the same
per-path create mutex on AdsOpenIndex; map sharing-violation to 7040."

Task 2: SetOrder iid 0 / missing is natural order, not 5000

Files:

  • Modify: src/network/session.cpp (Opcode::SetOrder ~2723–2736)
  • Modify: src/network/client.cpp (set_order, set_order_by_name, open_index Error-frame handling)

  • Test: tests/unit/abi_setorder_natural_remote_test.cpp (new)

  • Step 1: Write the failing test

Remote table, no index opened. Call AdsSetIndexOrderByHandle(hTable, 0) then AdsAppendRecord + AdsSetString + AdsWriteRecord. Must return 0 on SetOrder (natural) and the append must land.

Second case: open a table whose production CDX exists, but skip AdsOpenIndex; AdsSetIndexOrder to "IDX01" (or handle from GetIndexHandle) must either auto-open the bag or return a real not-found — never 5000. After a successful OpenIndex, SetOrder(0) still Acks.

  • Step 2: Run it — must FAIL
tests\ttest.cmd "*SetOrder*natural*"

Expected: FAIL, AdsSetIndexOrderByHandle / wire SetOrder returns 5000 (SetOrder: bad index id).

  • Step 3: Session SetOrder

Replace the missing-id branch:

case Opcode::SetOrder: {
    if (f.payload.size() < 8) { reply = err("SetOrder: bad payload"); break; }
    std::uint32_t tid = read_u32_le(f.payload.data());
    std::uint32_t iid = read_u32_le(f.payload.data() + 4);
    ADSHANDLE ht = ensure_abi_handle(tid);
    if (ht == 0) { reply = err("SetOrder: bad table id"); break; }
    if (iid == 0) {
        UNSIGNED32 rrc = AdsSetIndexOrder(ht, nullptr);
        if (rrc != 0) { reply = err("SetOrder", rrc); break; }
        ordered_tables_.erase(tid);
        sync_engine_cursor(tid);
        reply.opcode = Opcode::SetOrderAck;
        break;
    }
    auto iit = index_h_.find(iid);
    if (iit == index_h_.end()) {
        // Production bag retry: AdsOpenIndex on the table's .cdx, then
        // look up iid again. If still missing, err with AE_NOT_FOUND
        // (include/openads/error.h — use the existing not-found code
        // AdsOpenIndex already returns, not 5000).
        reply = err("SetOrder: unknown index id", openads::AE_NOT_FOUND);
        break;
    }
    UNSIGNED32 rrc = AdsSetIndexOrderByHandle(ht, iit->second);
    if (rrc != 0) { reply = err("SetOrder", rrc); break; }
    ordered_tables_.insert(tid);
    sync_engine_cursor(tid);
    reply.opcode = Opcode::SetOrderAck;
    break;
}

Look up the exact not-found constant in include/openads/error.h (AE_NOT_FOUND / AE_NO_MATCHING_FILE 5059 / AE_TABLE_NOT_FOUND 5066) and use the one AdsOpenIndex already returns for a missing bag. Do not invent a new code.

  • Step 4: Client keeps the ACE code from Error frames

RemoteConnection::set_order today:

if (rep.value().opcode != Opcode::SetOrderAck) {
    return util::Error{5000, 0, "SetOrder: server error", ""};
}

request() should already fail on Opcode::Error with the payload u32. If a nack still arrives as a non-Error opcode, read bytes 0–3 as the ACE code when payload size ≥ 4; only then fall back to 5000. Same for set_order_by_name and open_index.

  • Step 5: Re-run — PASS plus ttest.cmd "*SetOrder*" for unrelated SetOrder tests.

  • Step 6: Commit

git add src/network/session.cpp src/network/client.cpp tests/unit/abi_setorder_natural_remote_test.cpp tests/CMakeLists.txt
git -c user.name="Kimi3" -c user.email="mimo@opencode.ai" commit -m "SetOrder iid 0/missing is natural order, not 5000

B_BIG dbSetOrder(0) after a racing OpenIndex logged 30x AE_INTERNAL_ERROR
because session SetOrder treated a missing index id as err() default 5000."

Task 3: Append must hold a blocking record lock (no durable blanks)

Files:

  • Modify: src/engine/table.cpp (Table::append_record ~1148–1246)
  • Test: tests/unit/abi_append_lock_contention_test.cpp (new)

  • Existing must stay green: tests/unit/abi_append_autolock_test.cpp

  • Step 1: Write the failing test

32 local (or remote) connections, shared CDX table, no index required. Each: AdsAppendRecord, 4× AdsSetString on NAME/CITY/INS/RDD-like fields, AdsWriteRecord, AdsUnlockRecord. After join: physical scan — every record has non-blank NAME and INS; hdr_count == 32; AdsIsRecordLocked is 1 between append and unlock on a single-thread sanity case (already in autolock test).

To force contention: 32 threads tight-loop 20 appends each (640 rows). Fail the test if any record is all-spaces in NAME.

A second case: if we can inject lock failure, append must return 5012 and nrec must not grow. If injection is too heavy, skip and rely on the contention scan.

  • Step 2: Run it — must FAIL (or the 640-row scan shows blanks under current try_lock). If the local 32-thread case is already blank-free (lock bytes do not collide locally), run it remote through in-process Server so session AppendBlank + field puts match B_BIG. Remote is the one that showed 580 empty INS.

  • Step 3: Blocking lock in append_record

Replace:

(void)try_lock_record_excl(recno_);

with a blocking lock before the row is durable, preferred order:

  1. Under the driver header lock (already inside append_record_raw), recno is rec_count_+1. Taking the record lock there is ideal but crosses driver/engine. Do not split the header lock into a public API unless needed.
  2. Practical fix in Table::append_record: after append_record_raw returns recno_, call lock_record_excl(recno_) (blocking, table.cpp ~1833). If it fails, undo: rewrite header count to recno_-1 and put 0x1A at the previous EOF (driver helper or a new driver_->truncate_last_record()). Return the lock error (AE_LOCKED 5012).
  3. Never (void) the lock result.

If undo is messy, lock rec_count()+1 before append_record_raw via lock_record_excl(next) then append. On append failure, unlock. On success, the lock already covers the new recno.

  • Step 4: Confirm GoHot still rejects unlocked field puts

Do not weaken the 5035 guard. Partial REPLACE must not persist: if SetString fails, WriteRecord/Unlock must not leave NAME filled and INS empty from a later connection’s read of an unlocked blank — the lock held until Unlock prevents that.

  • Step 5: Re-run contention + autolock tests — PASS
tests\ttest.cmd "*append*autolock*" "*append*lock*contention*"
  • Step 6: Commit
git add src/engine/table.cpp tests/unit/abi_append_lock_contention_test.cpp tests/CMakeLists.txt
git -c user.name="Kimi3" -c user.email="mimo@opencode.ai" commit -m "Append waits for record lock; do not persist unlocked blanks

B_BIG N=700 left 580 INS-empty rows: Table::append_record ignored
try_lock_record_excl failure, so REPLACE ran (or stopped) on an
unlocked blank that was already in the DBF header count."

Task 4: No-barrier C++ storm + unmodified B_BIG × 700

Files:

  • Modify: tests/unit/abi_remote_create_stress_test.cpp — add a sibling TEST_CASE that does not wait on phase_b between INDEX ON and append (overlap OpenIndex/CreateIndex/Append like B_BIG). Keep the existing [slow] case unchanged.

  • Manual: C:\openads\_review_b_big\B_BIG.exe + current openace32.dll as ace32.dll, serverd + Pritpal INI.

  • Step 1: No-barrier C++ case, default 32 workers, [slow]

Same integrity scan as the existing test (header count, delete flags, size, three tag walks). Fail on any worker error other than a retried 7040 CreateTable. Run:

tests\ttest.cmd "*no-barrier*"

Must PASS at 32. Then OPENADS_STRESS_WORKERS=120 once locally.

  • Step 2: Rebuild x86 ACE + x64 serverd
cmake --build C:\openads\build\default --config Release --target openads_serverd -- /v:m /nologo
cmake --build C:\openads\build\x86 --config Release --target openads_ace -- /v:m /nologo
copy /Y C:\openads\build\x86\src\Release\openace32.dll C:\openads\_review_b_big\ace32.dll
  • Step 3: Wipe C:\Temp\TestFolder, start serverd with C:\Temp\openads.ini, spawn 700 minimized B_BIG

Reuse C:\openads\_review_b_big\storm700.py (raise TIMEOUT_S to 1800). Kill B_BIG after nrec stable.

  • Step 4: Two independent counts must agree

Header nrec and sequential walk == 70000; 0 invalid delete flags; 700 INS with 100 recs each; 0 blank NAME; CDX IDX01/02/03 present. ads_err.dbf may contain 7040 CreateTable; must not contain 5103 OpenIndex or 5000 SetOrder from this run.

If the C++ no-barrier storm is green but B_BIG still stalls, stop and report — do not paper over with PRG changes. Next lever is append wait-queue (spec non-goal C), not a client rewrite.

  • Step 5: CHANGELOG.md one entry, credit Pritpal Bedi. Commit with the test + changelog if B_BIG also passed; if only C++ passed, commit the test and leave B_BIG status in the message body.

Self-review

  • Spec patch 1 → Task 1 (mutex + 7040 map)
  • Spec patch 2 → Task 2 (SetOrder + client codes)
  • Spec patch 3 → Task 3 (blocking append lock)
  • Spec acceptance → Task 4 (no-barrier C++ + B_BIG 700)
  • No TBD. 7040 CreateTable left as SAP-correct. B_BIG.exe untouched.
This site uses Just the Docs, a documentation theme for Jekyll.