Rebirth: Starting from Lighting Up the Tech Tree

Chapter 113_Computing Power Solution

The next morning, Shen Yiming locked himself in the laboratory.

He barely slept last night, spending the entire night thinking about the specific implementation of collaborative reasoning. When Zuo Cheng arrived at the company at seven in the morning, the lab lights were already on.

"Yiming, have you been up all night?" Zuo Cheng pushed open the door and saw Shen Yiming asleep on the table.

Shen Yiming rubbed his eyes and sat up: "Brother Cheng, I've figured it out."

He picked up a stack of hand-drawn blueprints from the table: "Look, this is my design for a distributed edge AI architecture."

Zuo Cheng took the blueprints and carefully examined the lines and markings on them.

"Each satellite no longer runs a complete AI model, but rather a streamlined 'decision agent,'" Shen Yiming explained, pointing to the diagram. "The agent is only responsible for local decisions and communicates with other agents via inter-satellite links."

Fang Ze walked in at this moment, carrying three cups of coffee.

"Let me add some points about the hardware aspects," he said, putting down his coffee. "If we use a collaborative inference architecture, the computing power requirements of a single satellite can be significantly reduced. We can consider using low-power embedded AI chips."

"How low can the power consumption be reduced?" Zuo Cheng asked.

Fang Ze pulled up some test data: "I checked last night, and the lowest power consumption embedded AI chip in the industry can currently achieve around 2 watts. If three or four such chips work together, the power consumption is about 6 to 8 watts."

"It's still above the limit." Shen Yiming frowned. "The target is to reduce it to below 5 watts."

"Instead of using multiple low-power chips, let's compress the model of a single high-performance chip to its limit," Zuo Cheng suddenly said.

Shen Yiming's eyes lit up: "Model compression?"

"Yes." Zuo Cheng drew a diagram on the whiteboard. "If we can compress the size of the AI ​​model to one-thirtieth of its original size while maintaining a decision accuracy rate of over 80%, it can run on a single chip."

Fang Ze quickly calculated: "Cambricon's MLU270 consumes 30 watts. If it's compressed to one-thirtieth of that, theoretically, the power consumption could be reduced to about 1 watt."

"The question is, can our current compression technology do that?" Shen Yiming asked.

Zuo Cheng opened the system panel to view the current list of blades.

[Technology Leaf Piece]

- Neural Network Basics

- Deep learning framework

- Reinforcement learning strategies

Natural Language Understanding

Computer Vision

- Model compression optimization

- Federated Learning

- Causal reasoning

- Generative Model

- Gradient sparsity compression

Model compression optimization.

This leaf should be useful.

But he couldn't tell Shen Yiming these things directly.

"Yiming, do you think there's room for improvement in our compression technology?" Zuo Cheng asked.

Shen Yiming pondered for a moment: "Yes. But it requires a breakthrough in new algorithms."

"What kind of breakthrough?"

"There's a approach called 'knowledge distillation,'" Shen Yiming wrote on the whiteboard. "It involves 'distilling' knowledge from a large model into a smaller one. But the difficulty of knowledge distillation lies in the fact that we don't have a sufficiently powerful 'teacher model' to guide the student model."

Why?

"Because AI scheduling for Tianqiong satellites is a completely new scenario," Shen Yiming explained. "Traditional pre-trained models don't perform well in this scenario, so we need to train a dedicated large model from scratch."

How long will this take?

Shen Yiming estimated: "At least two to three months."

Three months to come.

They only had two weeks.

"There's another direction," Fang Ze suddenly said.

"Edge computing," Fang Ze said. "Instead of running a complete AI model on a satellite, it's better to move some of the computing tasks to ground stations or relay satellites."

"What do you mean?" Shen Yiming asked.

"Cloud-edge collaboration." Fang Ze drew a simple architecture diagram on the whiteboard. "Satellites are only responsible for simple perception and decision-making; complex reasoning is done on ground stations or relay satellites."

"But this will increase communication latency," Shen Yiming raised the question. "For scenarios with high real-time requirements, latency can be fatal."

"So we need to use predictive scheduling," Zuo Cheng suddenly said.

The two were taken aback.

"Our goal isn't for satellites to 'think in real time,' but rather to 'predict in advance,'" Zuo Cheng drew a timeline on the whiteboard. "For example, changes in the state of inter-satellite links are predictable. If a satellite can predict link congestion a few seconds in advance, it can adjust routes ahead of time and avoid problems."

"Predictive scheduling..." Shen Yiming muttered to himself, "This requires a deep understanding of the dynamic characteristics of satellite networks."

"Don't we have federated learning?" Zuo Cheng said. "We use federated learning to allow all satellites to share network status information, and then use AI to predict link changes."

"Brother Cheng, you mean, using federated learning to train a prediction model?"

"Yes." Zuo Cheng nodded. "This model doesn't need to run on satellites. It only needs to be trained at ground stations and then the prediction parameters are sent to each satellite. The satellites can then perform simple calculations based on the parameters."

Shen Yiming's eyes lit up.

"This ensures the accuracy of the predictions while reducing the satellite's computing power requirements," Fang Ze added excitedly.

"How much can it be compressed?" Zuo Cheng asked.

Shen Yiming estimated: "What used to require compression to one-thirtieth of its original value may now only require one-tenth. The accuracy can be improved from 70% to around 85%."

85%.

We are one step closer to the 90% target.

"But that's not enough," Zuo Cheng shook his head. "It has to be over 95%."

Shen Yiming hesitated, "95% is difficult to achieve with simple compression. Unless..."

"Unless there's a breakthrough in new algorithms," Shen Yiming said, "for example, automatic hyperparameter optimization."

Automatic hyperparameter optimization.

This is exactly what he needs.

"How much can it improve?" Zuo Cheng asked.

"Theoretically, it could increase by 5 to 10 percentage points," Shen Yiming said.

5 to 10 percentage points.

85% plus 10% equals 95%.

Zuo Cheng closed his eyes and took a deep breath.

He needs this technology.

How long will it take for the technological radar to cool down?

He opened the system panel and glanced at it: 8 hours remaining.

Four hours later, he will be able to scan Shen Yiming and obtain the technology for automatic hyperparameter optimization.

"Okay." Zuo Cheng opened his eyes. "It's settled then."

"It's settled?" Shen Yiming was somewhat surprised.

"First, train the prediction model using federated learning. Second, compress the model to one-tenth its original size. Third, use hyperparameter optimization to automatically improve accuracy."

Zuo Cheng wrote three keywords on the whiteboard.

"To be completed within two weeks."

Shen Yiming and Fang Ze exchanged a glance, both seeing the determination in each other's eyes.

"No problem," Shen Yiming said. "I'll start writing the technical solution right away."

"Leave the chip issue to me," Fang Ze said. "I'll contact Cambricon to see if they can provide some technical support."

The three of them split up and got busy again in the meeting room.

Zuo Cheng was the last to leave. He glanced at the countdown on the system panel.

8 hours.

Just wait another 8 hours.