Market Data & Drop Copy¶

Objective¶

Observe real-time market data with the current EduMatcher tools and understand where the engine's drop-copy feed fits into the architecture.

Background reading

Prerequisites¶

Chapters 01–12 completed.
Engine and at least one trader gateway active so events are visible.

Background¶

EduMatcher publishes two event streams from pm-engine:

PUB :5556 — primary market data: books, trades, session state, order events.
PUB :5557 — drop-copy feed: per-participant fill events for risk, compliance, and back-office consumers.

There is no separate pm-drop-copy process. The engine binds the drop-copy feed itself when it starts.

Exercise 1: Open a Live Book Viewer¶

Start a live viewer for AAPL:

pm-viewer --symbol AAPL --depth 10

In a trader gateway, place or cancel a resting order and watch the viewer update:

[TRADER01]> NEW|SYM=AAPL|SIDE=BUY|TYPE=LIMIT|QTY=100|PRICE=149.80|TIF=DAY

Checkpoint: pm-viewer changes when the book changes.

Exercise 2: Run the Cross-Gateway Order Monitor¶

Start the order monitor:

pm-orders

Now place orders from both TRADER01 and TRADER02. The monitor should show resting order state across gateways.

Checkpoint: pm-orders shows orders from multiple gateways.

Exercise 3: Capture Events with pm-audit¶

Start the audit logger in terminal mode:

pm-audit --terminal

Execute a trade:

[TRADER01]> NEW|SYM=AAPL|SIDE=BUY|TYPE=MARKET|QTY=100

Watch pm-audit print the resulting events. This is the easiest training-safe way to observe the event stream without writing a custom ZMQ subscriber.

Checkpoint: audit output shows the trade/order lifecycle events.

Exercise 4: Confirm the Drop-Copy Feed Is Bound¶

Restart pm-engine --verbose (and then your trader and operator consoles, and reopen the market) and look for this startup line in the engine output (preceded by the usual timestamp, level and logger name):

Drop copy PUB bound on port 5557

That confirms the drop-copy publisher is active. It is intended for external risk/compliance subscribers and publishes topics such as drop_copy.event.<gateway_id>.

Checkpoint: you can identify the drop-copy socket in engine startup output.

Exercise 5: Watch the Drop-Copy Feed with `pm-dc-spy`¶

pm-audit, pm-viewer and pm-orders all read the public feed on port 5556 — none of them show the drop-copy feed. EduMatcher ships two tools for that, and it is worth knowing both:

Tool	What it is	Use it for
`pm-dc-spy`	A CLI that subscribes to the engine's drop-copy PUB socket (5557)	Looking at the feed yourself, right now
`pm-dc-gwy`	A gateway process serving the DC1 protocol over TCP 5590	Letting an external system consume drop copy, the way a real back office would

Start the spy:

pm-dc-spy --format human

Then submit a trade from a trader console and watch the fill events appear.

Checkpoint: pm-dc-spy prints a fill event for a trade you caused, and you can say why an external consumer would use pm-dc-gwy on 5590 instead of subscribing to 5557 directly.

Understanding what the tools do for you is worth one level down. Connect a minimal ZMQ subscriber to the same socket pm-dc-spy reads:

The snippet needs the zmq Python package (pyzmq); in a container shell or a Poetry checkout use python/poetry run python, and with pipx use pipx runpip edumatcher's interpreter or pip install pyzmq in a virtual environment.

python - <<'PY'
import json
import zmq

ctx = zmq.Context()
sock = ctx.socket(zmq.SUB)
sock.connect("tcp://127.0.0.1:5557")
sock.subscribe(b"drop_copy.event.")

print("listening on drop_copy.event.*")
for _ in range(10):
    topic, payload = sock.recv_multipart()
    msg = json.loads(payload)
    print(topic.decode(), "seq=", msg.get("seq"), "gateway=", msg.get("gateway_id"),
            "event_type=", msg.get("event_type"), "trade_ids=", msg.get("trade_ids"))
PY

While this is running, execute a trade from TRADER01 in another terminal:

[TRADER01]> NEW|SYM=AAPL|SIDE=BUY|TYPE=MARKET|QTY=100

You should see one drop_copy.event.<gateway_id> message per participant on each side of the trade, stamped with a seq that counts that gateway's own events (each gateway has its own counter starting at 1, so the two sides of one trade carry numbers from different streams). Each message also carries trade_ids, the durable public trade ID that lets a risk or clearing consumer correlate its private execution with trade.executed.

Checkpoint: you received at least one drop-copy message directly from port 5557, distinct from anything pm-audit printed on port 5556.

Exercise 6: Compare Public Events and Drop-Copy Purpose¶

Execute another trade and compare what each consumer is for:

Consumer	Source	Purpose
`pm-viewer`	PUB :5556	Human-readable book view
`pm-orders`	PUB :5556	Cross-gateway resting order monitor
`pm-audit --terminal`	PUB :5556	Full event stream for inspection/logging
External drop-copy client	PUB :5557	Per-participant fill feed for risk/compliance

Checkpoint: explain why drop-copy is separate from the public market-data stream.

Exercise 7: Launch the Market Board¶

Start the multi-symbol dashboard:

pm-board --rows 8 --interval 10

pm-board is a pure market-data subscriber: it builds its table from the live book.* and trade.executed topics only and does not read stats.db. It auto-rotates pages every --interval seconds; Enter, PgUp and PgDn page manually.

Checkpoint: board shows AAPL, MSFT, and TSLA in one view.

Key Architecture¶

flowchart LR
    E[pm-engine]
    E -->|PUB :5556| V[pm-viewer / pm-orders / pm-audit / pm-stats]
    E -->|PUB :5557| DC[external drop-copy consumers]
    V --> B[pm-board / pm-ticker]

Reflection¶

Why does the engine publish drop-copy events on a separate socket (:5557) rather than mixing them into the same :5556 PUB feed that viewers, orders, audit, and stats all subscribe to? What operational problem would arise for a compliance drop-copy consumer if it shared a socket with high-volume book/viewer traffic?