top of page
Search

Skymizer's HyperThought: A Leap Towards Efficient Edge AI that Can Operate on 28nm chips

  • J L
  • May 10
  • 3 min read

Skymizer, a Taiwan-based semiconductor IP company, has recently unveiled its latest breakthrough technology, "HyperThought," an LPU (Language Processing Unit) IP that is generating significant buzz in the tech world. HyperThought is being hailed as a potential game-changer in the push for on-device AI, promising to bring the power of generative AI, currently seen in the cloud, to everyday devices.

Established 11 years ago, Skymizer spent its initial eight years focusing on compiler software. This deep dive into software has provided the company with a unique perspective and a strong foundation for its recent move into developing its own IP products. Their first IP, Edge Thought, launched just last year, paved the way for HyperThought, which represents a substantial leap forward in a short time.

"EdgeThought brings large language model inference to the edge, while HyperThought makes the edge smarter," said William Wei, CMO of Skymizer."With LISA v3, we’re not just accelerating tokens—we’re enabling intelligent agents with reasoning, multimodal interaction, and real-time adaptability. This marks a major milestone in the development of edge AI."

Founder and CTO Luba Tang said to the audience in a press conference to unveil the new product “HyperThought” confidently, "Compared to competitors requiring 100s of TOPS, we achieve comparable performance with only 0.5 TOPS. This demonstrates a significant advantage in PPAC (Performance, Power, Area, and Cost), as we use a less advanced 28nm process, leading to lower cost, smaller chip size, and lower power consumption, outperforming competitors by 100 to 1000x.”

Chairman Jim Lai emphasized that the metric for comparison should be Tokens Per Second Per Watt, not just raw power. While 4nm is inherently more power-efficient than 28nm, our 28nm reference chip delivers a higher Token Per Second Per Watt. Of course, if the IP is built into a 4nm chip, it would have even lower absolute power consumption.

HyperThought vs. Edge Thought: A Multimodal, Multi-Model Approach

While Edge Thought focused on bringing large language models (LLMs) to edge devices for text-based interactions, HyperThought is designed to overcome its limitations by incorporating multi-modality and the ability to run multiple models simultaneously.

  • Multi-model refers to HyperThought's capability to run several different AI models concurrently, such as those for speech, vision, and reasoning.

  • Multimodality refers to the ability of the models to process different types of data together, including text, voice, images, and sensor data.

This allows for more sophisticated and context-aware AI applications on devices.

LISA V3: The Language of Efficient AI

A key component of HyperThought is LISA V3, Skymizer's third-generation Language Instruction Set Architecture. LISA V3 is a specialized language that the HyperThought hardware “speaks,” optimized for efficient execution of complex AI tasks, multi-modality, and concurrent running of multiple models. Skymizer’s background in compiler software has been crucial in developing LISA V3, enabling a smart architecture that prioritizes efficiency.

Key Features and Benefits of HyperThought

  • Efficiency: HyperThought achieves high performance with lower resources through sophisticated compression techniques and tight hardware-software integration.

  • Performance: HyperThought delivers high tokens per second (TPS) with low tera operations per second (TOPS) and bandwidth, indicating efficient AI generation.

  • Cost-Effectiveness: HyperThought's design allows for the use of older, cheaper manufacturing processes and standard memory, reducing costs.

  • Hyperstating: This architectural feature enables multiple users or programs to run concurrently and securely on the same HyperThought chip, improving resource sharing and scalability.

  • Security: HyperThought incorporates built-in security features, including encryption and isolation of short-term memory, to protect against attacks and manipulation.

  • Tool Integration: HyperThought allows AI to access external information or control other functions, expanding its capabilities.

  • Open Source Model Support: HyperThought supports standard open-source models, providing flexibility for developers.

Applications and Future Directions

  • Robotics: Service robots and delivery robots are noted as current applications.

  • Smart Retail: Natural language interfaces in stores for smart retail applications.

  • Meeting Recorders: Smart meeting recorders for meeting integration.

  • NAS Systems: Video searching and retrieval in Network Attached Storage (NAS) systems.

  • Security: Data centers utilize Skymizer's technology for enhanced security.

  • Automotive: There's a focus on automotive, particularly for AI assistance and autonomous driving features.

  • Smart Devices: Potential future applications include smart glasses and earbuds for translation and information.

  • AI PCs: Enabling existing PCs to become AI PCs with external solutions.


 
 
 

Comments

bottom of page