Andrew Whigham

𝗝𝗲𝘁𝘀𝗼𝗻 𝗢𝗿𝗶𝗻 𝗡𝗮𝗻𝗼 𝘃𝘀 𝗝𝗲𝘁𝘀𝗼𝗻 𝗡𝗮𝗻𝗼 The goal of this work is to compare two developer boards for deploying neural networks and AI/computer vision workloads: the NVIDIA Jetson Nano and the NVIDIA Jetson Orin Nano. NVIDIA claims up to 𝟴𝟬× AI performance improvement, my goal was to benchmark both platforms and see how they really perform across practical workloads. 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗰𝗼𝗻𝗳𝗶𝗴𝘂𝗿𝗮𝘁𝗶𝗼𝗻 Jetson Nano is limited to 𝗝𝗲𝘁𝗣𝗮𝗰𝗸 𝟰.𝘅, while Orin Nano uses the latest 𝗝𝗲𝘁𝗣𝗮𝗰𝗸 𝟲.𝘅 with newer frameworks (TensorRT, Torchvision, TensorFlow). This software gap affects benchmarks but can’t be avoided. Power use is similar: Nano is rated at 𝟭𝟬 𝗪 but can peak near 𝟮𝟬 𝗪, while Orin Nano draws about the same under load. 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗠𝗲𝘁𝗵𝗼𝗱𝗼𝗹𝗼𝗴𝘆 1. 𝗦𝘆𝗻𝘁𝗵𝗲𝘁𝗶𝗰 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸𝘀: 7 models from the NVIDIA Jetson Inference set (ResNet-50, YOLOv3-tiny, etc.). 2. 𝗩𝗶𝗱𝗲𝗼 𝗱𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻: YOLOv5s script → frame extraction → annotation → FFmpeg re-encoding. 3. 𝗥𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗱𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻: 1080p30 USB camera, FPS output via DeepStream. 4. Each test was run 3 times on a cooled device; the median result was recorded. 5. Power mode was set to maximum (“𝗠𝗔𝗫𝗡”). 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗿𝗲𝘀𝘂𝗹𝘁𝘀 The average performance gain was 𝟮𝟲.𝟮𝟯× on real computer vision models. 𝗖𝗼𝗺𝗽𝗮𝗿𝗶𝘀𝗼𝗻 𝗼𝗳 𝗩𝗶𝗱𝗲𝗼 𝗙𝗶𝗹𝗲 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲. This test involves loading a pre-recorded video, slicing it into frames, processing each frame using 𝗬𝗢𝗟𝗢𝘃𝟱𝘀, drawing detection results, and reassembling the video. It provides a comprehensive evaluation of both platforms’ performance, incorporating 𝗦𝗦𝗗 𝘀𝗽𝗲𝗲𝗱, 𝗖𝗣𝗨, 𝗮𝗻𝗱 𝗚𝗣𝗨 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲. The test clearly demonstrates that 𝗝𝗲𝘁𝘀𝗼𝗻 𝗢𝗿𝗶𝗻 𝗡𝗮𝗻𝗼 performs about 𝟯 𝘁𝗼 𝟯.𝟱× 𝗳𝗮𝘀𝘁𝗲𝗿 in this complex task. However, in tasks focused solely on real-time video detection, the performance gap is much greater, as the load primarily falls on the GPU. 𝗥𝗲𝗮𝗹-𝗧𝗶𝗺𝗲 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗖𝗼𝗺𝗽𝗮𝗿𝗶𝘀𝗼𝗻 Jetson Orin Nano shows a much larger advantage in real-time tasks, outperforming Nano by 10×–40× thanks to stronger GPU power. 𝗣𝗿𝗶𝗰𝗲 𝗮𝗻𝗱 𝗖𝗼𝘀𝘁-𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 𝗶𝗻 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 The older-generation Jetson Nano retailed for around $𝟮𝟬𝟬–$𝟮𝟱𝟬, while the initial price of the 𝗢𝗿𝗶𝗻 𝗡𝗮𝗻𝗼 was $𝟱𝟬𝟬. However, in late December 2024, NVIDIA announced the 𝗝𝗲𝘁𝘀𝗼𝗻 𝗢𝗿𝗶𝗻 𝗡𝗮𝗻𝗼 𝗦𝘂𝗽𝗲𝗿 𝗖𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻 1. Real performance gain: ~26× (not 80×, higher possible with INT8). 2. 𝗣𝗼𝘄𝗲𝗿 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 improved by roughly 𝟮𝟯×, which is critical for autonomous robotics. 3. 𝗝𝗲𝘁𝘀𝗼𝗻 𝗡𝗮𝗻𝗼 still has a role in very lightweight or legacy JetPack 4.x-based projects. 4. For long-term products, Jetson Nano is not recommended: limited RAM and security patches will end by 2026.

2

A system should not be able to say: The answer fabricated evidence, but it was clear, fast, and well-formatted, so it passed. Focus on the system not the model! I recommend using CAP rules in your systems to help contain mathematically sound judgments but overall untrue outcomes. That is how bad AI governance gets dressed up as math. Create CAP rules to contain these errors. The definition of a CAP rule: CAP = Constraint-Activated Penalty A rule that limits the maximum score a metric can receive when a known failure condition is present. CAPs protect the equation from being gamed by strong performance in less important areas. Any defect that changes whether the output can be trusted should have a CAP or many CAPs. Any defect that makes the output unsafe should create a hard stop.

15

4 Comments

Rachit Jain's walkthrough shows how to profile kernels with PyTorch tools, import traces into NVIDIA Nsight or Intel Advisor, and read the slanted ceilings that separate compute-bound from memory-bound regions.

5

Another interesting new RL perspective. They propose Reinforced Attention Learning (RAL) to address the difficulty of applying RL techniques that work well for LLMs but tend to struggle more with multimodal LLMs, although I think the idea is more general than that. They argue that this approach is more robust to reward hacking and less prone to diversity collapse. In practice, they add to the standard RL objective an additional term that looks like: E[ A_t · D(p_t^θ || p_t^old) ] Here, A_t is the advantage, while p_t is the attention distribution policy at generation step t, defined from the final attention layer (averaged across heads): the current one (θ) and the one that generated the batch (old). p_t^θ(i) is simply the attention weight from the generated token at position t to a previous token at position i < t, normalized by the sum of attention weights before t. D is the Jensen–Shannon divergence (JSD), a symmetric and bounded divergence defined via the mixture of the two distributions, and less aggressive than Kullback-Leibler divergence (KL). JSD(P || Q) = 1/2 · KL(P || M) + 1/2 · KL(Q || M) With M = 1/2 · (P + Q). What is happening here is that we incentivize attention patterns associated with high advantage and discourage those associated with low advantage. As the authors put it, when A_t < 0 the model is explicitly pushed to explore new reasoning paths. The full objective becomes: L_RL + λ · L_AttnRL. They then introduce an extension that combines RAL with on-policy knowledge distillation and attention distillation. The idea is similar to on-policy distillation: you generate with the student (the current network), compute the teacher outputs on the same generation, and align them using a reverse KL on the logits. In addition, they also align the attention distributions between teacher and student, again using JSD. Unlike RAL, this attention distillation term does not use an advantage, it is pure structural alignment. Takeaway: RL and on-policy distillation do not have to operate only at the token level. Extending the training signal to internal structures like attention can be quite effective, and I expect this direction to be explored much more in the future. Thanks Petru-Daniel (Dan) Tudosiu for suggesting the paper!

85

1 Comment

Is the computational ceiling about to break? The "SAT Problem" is the invisible bottleneck of our era. From EDA chip design at Nvidia and Intel to the formal verification of AWS cloud security and the scaling of DeepMind’s next-gen AI—we are all running into the same wall: Exponential Complexity. Current solvers (CDCL/Local Search) are masterpieces of engineering, but they still "stall" when faced with high-treewidth, structured monoliths. At Digital Dynamics AI Inc., we have developed a new framework: tEPIC (Topological Exact Partitioning & Integrated Certification). We have moved past heuristic-based guessing. Our solver, SOSAT, utilizes two novel, peer-reviewed-grade theorems to navigate the logic landscape deterministically: 1️⃣ The Charging Inequality (GO-360-CH): A combinatorial guard that forces any SAT instance to reveal its internal "triage" state. Every sub-component is mathematically boxed into one of three exits: a global fracture (Separator), a governing bottleneck (Core), or a high-density collision (Cycle). 2️⃣ Semantic Splice-or-Pivot (GO-361): A proof-complexity compiler that serializes local Unit Propagation traces into globally verifiable DRAT/RUP certificates. This allows us to "splice" overlapping micro-proofs into a unified solution path in strictly polynomial time. The result? A structural engine that doesn't just "try" to solve a problem—it exhausts the instance's topology until a refutation or a witness is found. No metaphors. No "magic." Just discrete graph accounting. I am now looking to present the SOSAT/tEPIC architecture to the Automated Reasoning and AI Safety teams at Microsoft Research, Google DeepMind, and Amazon Web Services. We are ready to demonstrate performance on industrial-scale "unsolvable" monoliths via a secure, black-box audit. If you are a Principal Engineer or Researcher in Formal Methods or AI Scaling, let’s talk about moving the world past the exponential wall. #SAT #DeepTech #ComputerScience #AI #FormalMethods #DigitalDynamicsAI #PvsNP #Innovation

2

30 Comments