METASTABILITY AND CROSSING CLOCK DOMAINS IN AN FPGA

ABSTRACT

What is metastability? What is meant by the crossing clock domain? This paper looks at the basic notion of metastability and the reasons for its 'appearance'. This includes analyzing data, discovering metastable states, and uncovering the nature of their nature. This paper also looks at basic techniques for crossing clock domains. What is setup time and hold time and why it is important they are correctly followed. The FIFO concept is the best option for crossing different clock domains and avoiding metastable states. We will also examine the Mean Time Between Failures (MTBF) formula.

АННОТАЦИЯ

Что такое метастабильность? Как правильно пересечь тактовый домен? В этой статье рассматривается понятие метастабильности и причины ее появления, а также как избежать этого состояния. Рассматриваются основные техники пересечения тактового домена, понятия "setup time" и "hold time". FIFO как лучший вариант для пересечения тактового домена. Мы также рассмотрим формулу среднего времени наработки на отказ (MTBF).

Keywords: FPGA, crossing clock domain, metastability, flip flop, FIFO, MTBF. setup time, hold time.

Ключевые слова: FPGA, ПЛИС, метастабильность, регистр, MTBF, межрегистровая передача.

Introduction

If you are writing FPGA-based projects with frequencies less than 20MHz, you have probably not experienced any of the following problems described below. In any professional FPGA-based project after synthesis of the project, you will need to create a set of ‘time-limit’[1] files. Without these files, when the FPGA is running, you will not know for sure where in the project the data processing is interrupted. After synthesizing[2] our project, we get a file with physical connections of our elements inside the FPGA and a time analyzer embedded in our developer program is used to calculate how much time it takes to transfer signals inside of the FPGA. On the basis of this analysis, the developer can see where in the project there is time and frequency slack, and if it is unnecessary to increase the frequency of the clock domain. Different clock domains and their crossing within one FPGA is common problem, but many inexperienced digital circuit engineers do not know how to resolve this properly or in which cases to use a specific algorithm.

D - flip flop and metastable state

 When the clock signal changes from "0" to "1" on the input, the D flip flop stores the current data value on the input D, and sends it to the output Q. With any other value of the edge of the clock signal, the value on Q remains unchanged. An asynchronous reset signal can also be present on the triggers to reset the output D.

Figure 1. D - flip flop and truth table

For the correct operation of the d flip flop, there are two time intervals related to the data reception on the D channel. This time is needed to stabilize the output signal.

• Ts - setup time. The d-flip flop input signal D, must remain stable for a minimum setup time before the rising edge of the clock signal.
• Th - hold time. The minimum hold time after the rising edge of the clock signal.

Figure 2. Setup time and hold time

"The time window" of these two parameters in the vicinity of the rising clock signal is the time during which the signal D must be stable. If the d flip flop is stabilized within this "time window", it is guaranteed to capture the incoming data to input D and transmit it to output Q without failure. An important point to note is the time window Ts and Th are clearly defined by the FPGA manufacturers and is not decided by the engineer and is created by time analysis utilities. The main purpose of timing analysis is to calculate the timing slack for a pair of registers between which data is exchanged.

• Setup slack - time margin from the moment of stabilization and to the beginning of setup time.
• Hold slack - time margin at the end of the hold time and before the loss of stabilization.
• For asynchronous reset there is also "a time window" and it is known as recovery time and removal time. The signal Q on the d flip flop output comes with a certain time delay. If the Ts or Th "time window" is violated, the Q signal on the d flip flop output can go into an undefined state. Sometimes when "the time window " is violated on the output Q we receive a signal which comes to the input D, close to the rising edge of the clock signal and retains the old value or forms a new value, but we do not know what it is. In in the worst case we will get an oscillating output signal between high and low level, which means there is a delay in switching the output to a stable state. This state is called the metastable state at the output Q. After a certain time d flip flop will go into one of the stable states of 0 or 1, but when, and which one, is impossible to predict. The metastability of one d flip flop, can unsynchronized other parts of the project, because usually one d flip flop sends an output signal to several d flip flops simultaneously thus the system cannot extricate out of such a state independently without causing a general failure in the process. Given that such a situation may occur regularly, it is possible to estimate the probability of such an event. The following formula (1.1) explains time between failures:

Rt - time interval of protection against metastability, ideally tends to 0 for one D flip flop, is equal to the period of the clock signal minus the delay for setting and propagation for the second D flip flop, plus the period for each subsequent D flip flop.

τ is a technological parameter measured in picoseconds.

 - "The time window" of entry into metastability, measured in picoseconds.

 - Clock signal d flip flop.

 - frequency of data change from 0 to .

The most common causes of metastability.

• Violation of "the time windows" Ts and Th for signal stabilization.
• Asynchronous reset in d flip flop must be synchronized to the clock signal of the d flip flop itself where the reset occurs.
• Transmit asynchronous signals at a different frequency relative to the clock frequency of the receiving d flip flops.
• Wrong synchronous signal from other external devices. It is necessary to set the time characteristics of the incoming signals.

Figure 3. Clock domain crossing

When there is only one operating frequency for the whole project, it makes the job a lot easier, but in complex projects there are a lot of clock domains. Data from sensors uses its frequencies, SDRAM, reads and writes work on its frequencies, while data coming from external devices, works on other frequencies. All of these different frequencies feed into the FPGA, and all of it must be synchronized to avoid metastability problems.

A solution to the problem of metastability and synchronization Single-bit signal

To reduce metastability in the simplest case for a single-bit signal, we add two d flip flops at the frequency of the receiving first trigger. If the first trigger enters a metastable state, then with high probability, it will pass to the moment of fixation in the second trigger, and after the third trigger will the signal become stable, 0 or 1.

Figure 4. Single-bit signal stabilization circuit

Clock domain crossing

Clock domain crossing, the single-digit signal

To transfer a single-bit signal from one clock domain to another clock domain, we use the circuit below, pic 1.5. The clock of the first domain goes in the first register, and in the second two registers the clock of the second domain goes in. The first register of the second clock domain works as a synchronization register to minimize the number of metastable failures, and to give extra time to bring the signal to a stable state, so that it may safely be transferred to the new clock domain.

Figure 5. Single-bit signal. Clock domain crossing circuit

Clock domain crossing, data stream. (multi-bit signal)

To transfer a large amount of data from one clock domain to another, requires using a dual port memory[] which allows you to read and write data from different ports at different frequencies. You can build such a memory by using the ip catalog built into any FPGA design editor. We write data into the FIFO at the first clock domain and read it out at another frequency, in another clock domain. The only limitation is that we cannot write and read data on the same address at the same time. Also, do not read data from an empty FIFO, or write data onto a full FIFO. It can cause a system failure.

Figure 6. 2 port FIFO

Conclusion

In general, the fewer the clock signals in the project, the fewer the problems. The higher the frequency of the FPGA, the less chance of a metastable failure because "the time window" Ts and Th are less than slow FPGAs. When transferring from one clock domain to another, it is necessary to use synchronization registers, or even better, FIFO. This will avoid problems with metastability and synchronization. This adds a certain time delays to the project, but this must be taken into account. All external signals including the reset signal are asynchronous to our FPGA project and must be synchronized. One asynchronous signal is synchronized to the project clock once. All internal clock signals, such as PLL[4] or DSP[5] module signals, are synchronous to the timing analyzer, and do not require re-synchronization.

References:

1. Steve Kilts. Advanced FPGA Design: Architecture, Implementation, and Optimization. 1st Edition. Wiley-IEEE Press 2007. P.89 - 110.
2. Roger Woods, John McAllister, Gaye Lightbody, Ying Yi. FPGA-based Implementation of Signal Processing Systems. 2nd Edition. Wiley 2017. P. 120 – 145.
3. Doug Amos, Austin Lesea, Ren Richter. FPGA-Based Prototyping Methodology Manual: Best Practices in Design-For-Prototyping. Synopsys Press 2011. P.87 – 110.
4. Jim Ledin. Architecting High-Performance Embedded Systems: Design and build high-performance real-time digital systems based on FPGAs and custom circuits. Packt Publishing 2021. P.56 – 101.
5. Uwe Meyer-Baese. Embedded Microprocessor System Design using FPGAs 1st ed. Kindle Edition. Springer 2012. P. 135-150.
6. Clive Maxfield. The Design Warrior's Guide to FPGAs: Devices, Tools and Flows (Edn Series for Design Engineers). Newnes 2004. P. 89 – 95.
7. Philip Simpson. FPGA Design: Best Practices for Team-based Design.  Springer 2010. P. 120 – 156.
8. Juan José Rodriguez Andina, Eduardo de la Torre Arnanz, Maria Dolores Valdés Peña. FPGAs: Fundamentals, Advanced Features, and Applications in Industrial Electronics. CRC Press 2017. P. 35 – 87.
9. Wayne Wolf. FPGA-Based System Design (Prentice Hall Modern Semiconductor Design Series).  Prentice Hall 2004. P. 120 – 143.
10. Seetharaman Ramachandran. Digital VLSI Systems Design: A Design Manual for Implementation of Projects on FPGAs and ASICs Using Verilog.  Springer 2007. P. 102 – 136.
11. Ian Grout. Digital Systems Design with FPGAs and CPLDs. Newnes 2008. P 23 – 87.