The problem is that when I run mips-android on qemu, it hangs when executing init program in the initramfs root file-system. Then I use the remote gdb to debug the init and finds out that it it because pa_workspace is not initiated.

Function ashmem_create_region will open /dev/ashmem and return the fd if succeed. However, it returns -1 and the errno is 19 which means NO SUCH DEVICES.

fd = ashmem_create_region("system_properties", size);

The problem is who is responsible for creating /dev/ashmem?

In fact, in android it uses udev mechanism to create devices in /dev when executing function device_init. The full cold patch to create a device is as following:

device_init->coldboot->do_coldboot->write(fd, "add\n", 4)->handle_device_fd->handle_device_event->make_device

In function parse_event, it will parse the uevent msg and then pass uevent to handle_device_event. However, I find that the uevent message is a little weird. I use remote gdb to dump this message.

0x7ff3d250:     "add@/class/tty/console"
0x7ff3d267:     "ACTION=add"
0x7ff3d272:     "DEVPATH=/class/tty/console"
0x7ff3d28d:     "SUBSYSTEM=tty"
0x7ff3d29b:     "MAJOR=+"
0x7ff3d2a3:     "MINOR=/"
0x7ff3d2ab:     "SEQNUM=31+"
0x7ff3d2b6:     ""
0x7ff3d2b7:     ""
0x7ff3d2b8:     ""

You see, in the message the MAJOR is +. It will confuse the parse_event so that the corresponding device won't be created.

It looks the kernel passes wrong uevent message to user space. So the question is who has messed up the uevent message?

Then I recall that when booting linux kernel, there are some weird messages.

Primary instruction cache .kB, VIPT, 2-way, linesize 1* bytes.
Primary data cache .kB, 2-way, VIPT, no aliases, linesize 1* bytes

You see the instruction cache is .kB and linesize is 1* bytes, not a valid number at all.

Then I suspect that something is wrong in kernel when parsing the numbers. So I use the remote gdb to debug the kernel again.

r4k_cache_init->probe_pcache->printk->vprintk->vscnprintf->vsnprintf->number->put_dec->put_dec_trunc

Function put_dec_trunc uses a unsigned int between[0,99999] as input and outputs the number as a string. But I find that when the input is 2, the output is '.', not the expected '2'. So maybe this function is the bad boy.

In the following function I assume q=2.

277 static char* put_dec_trunc(char *buf, unsigned q)
278 {
279         unsigned d3, d2, d1, d0;
280         d1 = (q>>4) & 0xf;            /*d1=0*/
281         d2 = (q>>8) & 0xf;            /*d2=0*/
282         d3 = (q>>12);                 /*d3=0*/
283
284         d0 = 6*(d3 + d2 + d1) + (q & 0xf);  /*d0=2*/
285         q = (d0 * 0xcd) >> 11;              /*q=0*/
286         d0 = d0 - 10*q;                     /*d==2*/
287         *buf++ = d0 + ''; /* least significant digit */
288         d1 = q + 9*d3 + 5*d2 + d1;   /*d1=0*/
289         if (d1 != 0) {               /* it is false so we won't get into here.*/
290                 q = (d1 * 0xcd) >> 11;
291                 d1 = d1 - 10*q;
292                 *buf++ = d1 + ''; /* next digit */
293
294                 d2 = q + 2*d2;
295                 if ((d2 != 0) || (d3 != 0)) {
296                         q = (d2 * 0xd) >> 7;
297                         d2 = d2 - 10*q;
298                         *buf++ = d2 + ''; /* next digit */
299
300                         d3 = q + 4*d3;
301                         if (d3 != 0) {
302                                 q = (d3 * 0xcd) >> 11;
303                                 d3 = d3 - 10*q;
304                                 *buf++ = d3 + '';  /* next digit */
305                                 if (q != 0)
306                                         *buf++ = q + '';  /* most sign. digit */
307                         }
308                 }
309         }    
310         return buf;
311 }

/*  MIPS uses a0/a1 to pass arguments.
 *  a0= address of bu
 *  a1= q = 2
 */
8015a040 <put_dec_trunc>:
8015a040:    00051202     srl    v0,a1,0x8        /* v0= q>>8*/
8015a044:    00053102     srl    a2,a1,0x4        /* a2= q>>4*/
8015a048:    30c6000f     andi    a2,a2,0xf        /* a2= (q>>4) & 0xf = d1 in line 280*/
8015a04c:    3048000f     andi    t0,v0,0xf        /* t0 = (q>>8) & 0xf = d2 in line 281*/
8015a050:    00054b02     srl    t1,a1,0xc        /* t1= (q>>12) = d3 in line 282*/
8015a054:    00c81021     addu    v0,a2,t0        
8015a058:    00491021     addu    v0,v0,t1          /* v0 = d1+d2+d3 in line 284*/
8015a05c:    24030006     li    v1,6
8015a060:    70433802     mul    a3,v0,v1          /*a3= 6*(d3 + d2 + d1)*/
8015a064:    30a5000f     andi    a1,a1,0xf         /*a1= q & 0xf*/
8015a068:    24020009     li    v0,9               /*v0=9*/
8015a06c:    00e56021     addu    t4,a3,a1          /*t4= 6*(d3 + d2 + d1) + (q & 0xf) = d0 in line 284*/
8015a070:    240b00cd     li    t3,205            /*t3= 0xcd*/
8015a074:    71223802     mul    a3,t1,v0          /*a3= 9*d3 in line 288. Apparently gcc has reordered the code.*/   
8015a078:    718b1802     mul    v1,t4,t3          /*v1= (d0*0xcd)*/
8015a07c:    24020005     li    v0,5
8015a080:    00e62821     addu    a1,a3,a2          /*a1= 9*d3 + d1 in line 288*/
8015a084:    71023002     mul    a2,t0,v0          /*a2= 5*d2 in line 288*/
8015a088:    00031ac2     srl    v1,v1,0xb         /*v1= (d0*0xcd)>>11 in line 285*/
8015a08c:    240a000a     li    t2,10             /*t2=10*/
8015a090:    01800013     mtlo    t4                /*put t4->lo. lo=t4= 6*(d3 + d2 + d1) + (q & 0xf) = d0 in line 284 */
8015a094:    706a0004     msub    v1,t2             /*hilo = v1*t2 - hilo = 10*(q) - hilo in line 286*/
8015a098:    00c51021     addu    v0,a2,a1
8015a09c:    00002812     mflo    a1                /*lo->a1*/    
8015a0a0:    00431821     addu    v1,v0,v1
8015a0a4:    24a20030     addiu    v0,a1,48         
8015a0a8:    00803821     move    a3,a0
8015a0ac:    a0820000     sb    v0,0(a0)

We just need to see the instruction in 0x8015a094, it is a msub instruction. The defination of msub is as following:

(HI,LO) = (HI,LO) - (GRP[RS]*GPR[RT])

Then after executing the instruction in 0x8015a094, the HI/LO should be 0/2. But qemu produces the value 0xffffffff/0xfffffffe, which is -2 indeed. Maybe this is the problem.

Then I need to find how qemu emulates msub instruction.

2178     case OPC_MSUB:
2179         {
2180             TCGv r_tmp1 = tcg_temp_new(TCG_TYPE_I64);
2181             TCGv r_tmp2 = tcg_temp_new(TCG_TYPE_I64);
2182             TCGv r_tmp3 = tcg_temp_new(TCG_TYPE_I64);
2183
2184             tcg_gen_ext32s_tl(t0, t0);
2185             tcg_gen_ext32s_tl(t1, t1);
2186             tcg_gen_ext_tl_i64(r_tmp1, t0);
2187             tcg_gen_ext_tl_i64(r_tmp2, t1);
2188             tcg_gen_mul_i64(r_tmp1, r_tmp1, r_tmp2);  /*r_tmp1= gpr[rs]*gpr[rt] */
2189             gen_load_LO(t0, 0);                /*t0 <- lo*/
2190             gen_load_HI(t1, 0);                /*t1 <- hi*/
2191             tcg_gen_extu_tl_i64(r_tmp2, t0);   /*r_tmp2 = 64bit expand of lo*/
2192             tcg_gen_extu_tl_i64(r_tmp3, t1);   /*r_tmp3 = 64bit expand of hi*/
2193             tcg_gen_shli_i64(r_tmp3, r_tmp3, 32);
2194             tcg_gen_or_i64(r_tmp2, r_tmp2, r_tmp3);
2195             tcg_temp_free(r_tmp3);
2196             tcg_gen_sub_i64(r_tmp1, r_tmp1, r_tmp2); /*r_tmp1= r_tmp1 - r_tmp2 = gpr[rs]*gpr[rt] - HI/LO */
2197             tcg_temp_free(r_tmp2);
2198             tcg_gen_trunc_i64_tl(t0, r_tmp1);
2199             tcg_gen_shri_i64(r_tmp1, r_tmp1, 32);
2200             tcg_gen_trunc_i64_tl(t1, r_tmp1);
2201             tcg_temp_free(r_tmp1);
2202             tcg_gen_ext32s_tl(t0, t0);
2203             tcg_gen_ext32s_tl(t1, t1);
2204             gen_store_LO(t0, 0);
2205             gen_store_HI(t1, 0);
2206         }
2207         opn = "msub";
2208         break;

You see, qemu makes an error emulation of msub instruction. It uses gpr[rs]*gpr[rt]-HI/LO and then put the results to HI/LO, which is different from the defination of msub instruction. I patched the qemu code and it works.

BTW: MIPS32 4KTM Processor Core Family Software User’s Manual version MD00016 gives an error operation of msub instruction on papge 253.

Operation:
   temp ← (HI || LO) - (GPR[rs] * GPR[rt])
   HI ← temp63..32
   LO ← temp31..0

Maybe this misleads the qemu developers. The latest qemu version has fixed this bug. We can see this instruction emulation in qemu-svn-20091014.

2195     case OPC_MSUB:
2196         {
2197             TCGv_i64 t2 = tcg_temp_new_i64();
2198             TCGv_i64 t3 = tcg_temp_new_i64();
2199
2200             tcg_gen_ext_tl_i64(t2, t0);
2201             tcg_gen_ext_tl_i64(t3, t1);
2202             tcg_gen_mul_i64(t2, t2, t3);  /*t2=GPR[RS]*GPR[RT] */
2203             tcg_gen_concat_tl_i64(t3, cpu_LO[0], cpu_HI[0]);  /*t3= HI/LO*/
2204             tcg_gen_sub_i64(t2, t3, t2); /*t2= HI/LO - GPR[RS]*GPR[RT] */
2205             tcg_temp_free_i64(t3);
2206             tcg_gen_trunc_i64_tl(t0, t2);
2207             tcg_gen_shri_i64(t2, t2, 32);
2208             tcg_gen_trunc_i64_tl(t1, t2);
2209             tcg_temp_free_i64(t2);
2210             tcg_gen_ext32s_tl(cpu_LO[0], t0);
2211             tcg_gen_ext32s_tl(cpu_HI[0], t1);
2212         }
2213         opn = "msub";
2214         break;

See this link for more information.

So finding this bug is really not easy. I have to dig dig and dig from userland to linux kernel and then to qemu until catching this bad qemu bug. Thanks to the remote gdb and gdb stub in qemu, it makes life easier.

Following is the patch of qemu.

diff --git a/target-mips/translate.c b/target-mips/translate.c
index 3dded6c..0a1b461 100644
--- a/target-mips/translate.c
+++ b/target-mips/translate.c
@@ -2193,7 +2193,11 @@ static void gen_muldiv (DisasContext *ctx, uint32_t opc,
             tcg_gen_shli_i64(r_tmp3, r_tmp3, 32);
             tcg_gen_or_i64(r_tmp2, r_tmp2, r_tmp3);
             tcg_temp_free(r_tmp3);
-            tcg_gen_sub_i64(r_tmp1, r_tmp1, r_tmp2);
+            /* msub means HI/LO = HI/LO - GPR[RS]*GPR[RT],
+             * not HI/LO = GPR[RS]*GPR[RT] - HI/LO
+             */
+            //tcg_gen_sub_i64(r_tmp1, r_tmp2, r_tmp2);
+            tcg_gen_sub_i64(r_tmp1, r_tmp2, r_tmp1);
             tcg_temp_free(r_tmp2);
             tcg_gen_trunc_i64_tl(t0, r_tmp1);
             tcg_gen_shri_i64(r_tmp1, r_tmp1, 32);

The implemention of memory watchpoint is tricky in qemu. In last article of qemu internal, we know that when emulating memory access, qemu needs to distinguish the normal RAM read/write from memory mapped I/O read/write. If it is a memory mapped I/O address access, qemu will dispatch this access to the registered I/O emulation functions. Qemu use this mechanism to implement the memory watchpoint. When accessing the memory address watched by qemu, qemu will dispatch this access to the registered memory watch functions, even if this address is normal guest RAM address or memory mapped I/O address! Qemu will do all the magic things in these memory watch functions.

In the following, I will use an example to explain the whole process of memory watch implement of qemu.

80103c60 <memcpy>:
80103c60:       00801021        move    v0,a0
80103c64 <__copy_user>:
80103c64:       2cca0004        sltiu   t2,a2,4
80103c68:       30890003        andi    t1,a0,0x3
80103c6c:       15400068        bnez    t2,80103e10 <__copy_user+0x1ac>
80103c70:       30a80003        andi    t0,a1,0x3
80103c74:       1520003d        bnez    t1,80103d6c <__copy_user+0x108>
80103c78:       00000000        nop
80103c7c:       15000046        bnez    t0,80103d98 <__copy_user+0x134>
80103c80:       00064142        srl     t0,a2,0x5
80103c84:       11000017        beqz    t0,80103ce4 <__copy_user+0x80>
80103c88:       30d8001f        andi    t8,a2,0x1f
80103c8c:       00000000        nop
80103c90:       8ca80000        lw      t0,0(a1)

These asm lines are objdumped from linux 2.6.30 kernel for mips malta. Assume that I want to  watch the memory access of virtual address 0x804cd000(swapper_pg_dir in linux kernel).

First I insert the watchpoint into cpu.

cpu_watchpoint_insert(env, 0x804cd000, 4, BP_GDB | BP_MEM_ACCESS,
                        NULL);

And then I need to register the vm state changing call back functions.

qemu_add_vm_change_state_handler(spy_vm_state_change, NULL);

If register a1=0x804cd000, guest linux kernel will touch the watched memory region when pc is 0x80103c90, then qemu dispatches this access to the registered memory watch function, even if this access is a noram guest RAM access.The memory watch functions in qemu are in array watch_mem_read/watch_mem_write.

exec.c

2649 static CPUReadMemoryFunc *watch_mem_read[3] = {
2650     watch_mem_readb,
2651     watch_mem_readw,
2652     watch_mem_readl,
2653 };
2654
2655 static CPUWriteMemoryFunc *watch_mem_write[3] = {
2656     watch_mem_writeb,
2657     watch_mem_writew,
2658     watch_mem_writel,
2659 };

In function watch_mem_readl, it will call function check_watchpoint first.

exec.c

2622 static uint32_t watch_mem_readl(void *opaque, target_phys_addr_t addr)
2623 {
2624     check_watchpoint(addr & ~TARGET_PAGE_MASK, ~0x3, BP_MEM_READ);
2625     return ldl_phys(addr);
2626 }

2563 static void check_watchpoint(int offset, int len_mask, int flags)
2564 {
2565     CPUState *env = cpu_single_env;
2566     target_ulong pc, cs_base;
2567     TranslationBlock *tb;
2568     target_ulong vaddr;
2569     CPUWatchpoint *wp;
2570     int cpu_flags;
2571
2572     if (env->watchpoint_hit) {
2573         /* We re-entered the check after replacing the TB. Now raise
2574          * the debug interrupt so that is will trigger after the
2575          * current instruction. */
2576         cpu_interrupt(env, CPU_INTERRUPT_DEBUG);
2577         return;
2578     }
2579     vaddr = (env->mem_io_vaddr & TARGET_PAGE_MASK) + offset;
2580     TAILQ_FOREACH(wp, &env->watchpoints, entry) {
2581         if ((vaddr == (wp->vaddr & len_mask) ||
2582              (vaddr & wp->len_mask) == wp->vaddr) && (wp->flags & flags)) {
2583             wp->flags |= BP_WATCHPOINT_HIT;
2584             if (!env->watchpoint_hit) {
2585                 env->watchpoint_hit = wp;
2586                 tb = tb_find_pc(env->mem_io_pc);
2587                 if (!tb) {
2588                     cpu_abort(env, "check_watchpoint: could not find TB for "
2589                               "pc=%p", (void *)env->mem_io_pc);
2590                 }
2591                 cpu_restore_state(tb, env, env->mem_io_pc, NULL);
2592                 tb_phys_invalidate(tb, -1);
2593                 if (wp->flags & BP_STOP_BEFORE_ACCESS) {
2594                     env->exception_index = EXCP_DEBUG;
2595                 } else {
2596                     cpu_get_tb_cpu_state(env, &pc, &cs_base, &cpu_flags);
2597                     tb_gen_code(env, pc, cs_base, cpu_flags, 1);
2598                 }
2599                 cpu_resume_from_signal(env, NULL);
2600             }
2601         } else {
2602             wp->flags &= ~BP_WATCHPOINT_HIT;
2603         }
2604     }
2605 }

When check_watchpoint is executed in the first time, env->watchpoint_hit is null. Then it will check whether the address is a watched address. If so, set the flag BP_WATCHPOINT_HIT in wp->flags(line 2583) and set env->watchpoint_hit to wp. Then it will find and invalidate the current translation block(line 2586-2592). If the flag BP_STOP_BEFORE_ACCESS in wp is not set, then qemu will translate the code from current pc(line 2596-2597) and resume the guest instruction emulation(line 2599). Function cpu_resume_from_signal will jump to line 256 in cpu-exec.c and rerun the emulation process from the lw instruction(pc=0x80103c90).

cpu-exec.c

255     for(;;) {
256         if (setjmp(env->jmp_env) == 0) {
257             env->current_tb = NULL;
258             /* if an exception is pending, we execute it here */
259             if (env->exception_index >= 0) {
260                 if (env->exception_index >= EXCP_INTERRUPT) {
261                     /* exit request from the cpu execution loop */
262                     ret = env->exception_index;
263                     if (ret == EXCP_DEBUG)
264                         cpu_handle_debug_exception(env);
265                     break;
266                 } else {

Why do qemu need to invalidate current translation block and regenerate the code? Because this memory access(pc=0x80103c90) is in the middle of a translation block. If we want to rerun this instruction, we need to regenerate the code from this instruction(pc=0x80103c90). Moreover before invalidating the translation block, qemu needs to sync the cpu state to guest cpu(cpu_restore_state). That’s because the cpu state in the middle of translation block is different from the actual cpu state. Understanding this process needs some knowledge of binary translation. If you find it is hard to understand, just ignore it.

Now qemu rerun the guest os from pc=0x80103c90. Because the memory address is a watched memory address, qemu will call watch_mem_readl->check_watchpoint again. But this time, env->watchpoint_hit is not null(qemu set it in last call), then it will call cpu_interrupt and return from function check_watchpoint. Then in watch_mem_readl it will call ldl_phys to fetch the value from guest RAM. Function cpu_interrupt in check_watchpoint  sets the CPU_INTERRUPT_DEBUG to flag to env->interrupt_request. 

Then qemu runs normally just like nothing has happened. Because the CPU_INTERRUPT_DEBUG has been set in env->interrupt_request, the main loop of cpu emulation will return.

cpu-exec.c

355                     if (interrupt_request & CPU_INTERRUPT_DEBUG) {
356                         env->interrupt_request &= ~CPU_INTERRUPT_DEBUG;
357                         env->exception_index = EXCP_DEBUG;
358                         cpu_loop_exit();
359                     }

54 void cpu_loop_exit(void)
55 {
56     /* NOTE: the register at this point must be saved by hand because
57        longjmp restore them */
58     regs_to_env();
59     longjmp(env->jmp_env, 1);
60 }

Function cpu_loop_exit will do longjmp to line 256 in cpu-exec.c. Because env->exception_index is EXCP_DEBUG, it will break from the loop of function cpu_exec. Function cpu_exec returns to main_loop in vl.c.

vl.c

3800                 ret = cpu_exec(env);

3850             if (unlikely(ret == EXCP_DEBUG)) {
3851                 gdb_set_stop_cpu(cur_cpu);
3852                 vm_stop(EXCP_DEBUG);
3853             }

It will call gdb_set_stop_cpu and then vm_stop to stop the qemu. It the virtual state is changed, qemu will the call the callback functions registered by qemu_add_vm_change_state_handler. So the function spy_vm_state_change will be called.

In sum, when accessing the watched memory address, the memory watch functions will be called. It will call function check_watchpoint. Function check_watchpoint will set env->watchpoint_hit to current watchpoint and rerun the guest os/applicaton from current pc. Then memory watched functions will be called again. It will call function check_watchpoint. This time, function check_watchpoint just set the flag in env->interrupt_request and tells cpu to interrupt the emulation process. And then qemu will return to the main_loop and stop the vm. At last it will call the registered vm change state callback functions.

1. the two level guest physical page descriptor table

Qemu uses a two level guest physical page descriptor table to maintain the guest memory space and MMIO space. The table is pointed by l1_phys_map. Bits [31:22] is used to index first level entry and bits [21:12] is used to index the second level entry. The entry of the second level table is PhysPageDesc.

exec.c

146 typedef struct PhysPageDesc {
147     /* offset in host memory of the page + io_index in the low bits */
148     ram_addr_t phys_offset;
149     ram_addr_t region_offset;
150 } PhysPageDesc;

If the memory region is RAM, then the bits [31:12] of phys_offset means the offset of this page in emulated physical memory. If the memory region is memory mapped I/O, then the bits of [11:3] of phys_offset means the index in io_mem_write/io_mem_read array. When accessing this memory region, the functions in io_mem_write/io_mem_read of index phys_offset will be called.

2. register the guest physical memory

Function cpu_register_physical_memory is used to register a guest memory region. If phys_offset is IO_MEM_RAM then it means this region is guest RAM space. If the phys_offset >IO_MEM_ROM, then it means this memory region is MMIO space.

898 static inline void cpu_register_physical_memory(target_phys_addr_t start_addr,
899                                                 ram_addr_t size,
900                                                 ram_addr_t phys_offset)
901 {
902     cpu_register_physical_memory_offset(start_addr, size, phys_offset, 0);
903 }

Function cpu_register_physical_memory_offset will first find the PhysPageDesc in table l1_phys_map using the given guest physical address. If finding the entry, qemu will update the entry. If not finding the entry, then qemu creates a new entry and updates its value and insert this entry to the table at last.

In malta emulation, the following is the code to register malta RAM space.

hw/mips_malta.c

811     cpu_register_physical_memory(0, ram_size, IO_MEM_RAM);

3. register the mmio space

Before registering mmio space using cpu_register_physical_memory, qemu uses the function cpu_register_io_memory to register the I/O emulation functions to array io_mem_write/io_mem_read.

exec.c

2851 int cpu_register_io_memory(int io_index,
2852                            CPUReadMemoryFunc **mem_read,
2853                            CPUWriteMemoryFunc **mem_write,
2854                            void *opaque)

This function will return the index in array io_mem_write/io_mem_read and this index will be passed to function cpu_register_physical_memory via parameter phys_offset.

hw/mips_malta.c

malta = cpu_register_io_memory(0, malta_fpga_read,
                                   malta_fpga_write, s);

cpu_register_physical_memory(base, 0x900, malta);

4. softmmu

Given the guest virtual address, how does qemu find the corresponding host virtual address? First qemu needs to translate the guest virtual address to guest physical address. Then qemu needs to find the PhysPageDesc entry in table l1_phys_map and get the phys_offset. At last qemu should add phys_offset to phys_ram_base to get the host virtual address.

Qemu uses a softmmu model to speed up this process. Its main idea is storing the offset of guest virtual address to host virtual address in a TLB table. When translating the guest virtual address to host virtual address, it will search this TLB table firstly. If there is an entry in the table, then qemu can add this offset to guest virtual address to get the host virtual address directly. Otherwise, it needs to search the l1_phys_map table and then fill the corresponding entry to the TLB table. The index of this TLB table is bits [19:12] of guest virtual address and there is no asid field in tlb entry. This means the TLB table needs to be flushed in process switch!

This TLB table idea is just like the most traditional hardware TLB. However, to MIPS cpu, there is another mmu model in qemu. Unlike x86 cpu, MIPS does NOT care about hardware page table. Instead it uses hardware TLB which is NOT transparent to software. Maybe It is another topic I will explain in another article. What we need to understand here is that the softmmu model in this article is not the mmu model of MIPS cpu itself.

Moreover, besides helping speed up the process of translating guest virtual address to host virtual address, this softmmu model can speed up the process of dispatching I/O emulation functions according to guest virtual address too. In this case, the idex of I/O emulation functions in io_mem_write/io_mem_read is stored in iotlb.

The format of TLB entry is as flowing:

cpu-defs.h

176     CPUTLBEntry tlb_table[NB_MMU_MODES][CPU_TLB_SIZE];                  \
177     target_phys_addr_t iotlb[NB_MMU_MODES][CPU_TLB_SIZE];  

108 typedef struct CPUTLBEntry {
109     /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address
110        bit TARGET_PAGE_BITS-1..4  : Nonzero for accesses that should not
111                                     go directly to ram.
112        bit 3                      : indicates that the entry is invalid
113        bit 2..0                   : zero
114     */
115     target_ulong addr_read;
116     target_ulong addr_write;
117     target_ulong addr_code;
124     target_phys_addr_t addend;
131 } CPUTLBEntry;

Field addr_read/write/code stores the guest virtual address for TLB entry. It is the tag of this entry. Filed addend is the offset of host virtual address to guest virtual address. We can add this value to guest virtual address to get the host virtual address.

addend = host_virtual_address – guest_virtual_address

host_virtual_address = phys_ram_base(qemu variable) + guest_physical_address – guest_physical_address_base(0 in MIPS)

The iotlb stores the index of I/O emulation function in io_mem_write/io_mem_read.

Function __ldb_mmu/__ldl_mmu/__ldw_mmu is used to translating the guest virtual address to host virtual address or dispatching guest virtual address to I/O emulation functions.

softmmu_template.h

86 DATA_TYPE REGPARM glue(glue(__ld, SUFFIX), MMUSUFFIX)(target_ulong addr,
87                                                       int mmu_idx)
88 {
89     DATA_TYPE res;
90     int index;
91     target_ulong tlb_addr;
92     target_phys_addr_t addend;
93     void *retaddr;
94
95     /* test if there is match for unaligned or IO access */
96     /* XXX: could done more in memory macro in a non portable way */
97     index = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1);
98  redo:
99     tlb_addr = env->tlb_table[mmu_idx][index].ADDR_READ;
100     if ((addr & TARGET_PAGE_MASK) == (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) {
101         if (tlb_addr & ~TARGET_PAGE_MASK) {
102             /* IO access */
103             if ((addr & (DATA_SIZE - 1)) != 0)
104                 goto do_unaligned_access;
105             retaddr = GETPC();
106             addend = env->iotlb[mmu_idx][index];
107             res = glue(io_read, SUFFIX)(addend, addr, retaddr);
108         } else if (((addr & ~TARGET_PAGE_MASK) + DATA_SIZE - 1) >= TARGET_PAGE_SIZE) {
109             /* slow unaligned access (it spans two pages or IO) */
110         do_unaligned_access:
111             retaddr = GETPC();
112 #ifdef ALIGNED_ONLY
113             do_unaligned_access(addr, READ_ACCESS_TYPE, mmu_idx, retaddr);
114 #endif
115             res = glue(glue(slow_ld, SUFFIX), MMUSUFFIX)(addr,
116                                                          mmu_idx, retaddr);
117         } else {
118             /* unaligned/aligned access in the same page */
119 #ifdef ALIGNED_ONLY
120             if ((addr & (DATA_SIZE - 1)) != 0) {
121                 retaddr = GETPC();
122                 do_unaligned_access(addr, READ_ACCESS_TYPE, mmu_idx, retaddr);
123             }
124 #endif
125             addend = env->tlb_table[mmu_idx][index].addend;
126             res = glue(glue(ld, USUFFIX), _raw)((uint8_t *)(long)(addr+addend));
127         }
128     } else {
129         /* the page is not in the TLB : fill it */
130         retaddr = GETPC();
131 #ifdef ALIGNED_ONLY
132         if ((addr & (DATA_SIZE - 1)) != 0)
133             do_unaligned_access(addr, READ_ACCESS_TYPE, mmu_idx, retaddr);
134 #endif
135         tlb_fill(addr, READ_ACCESS_TYPE, mmu_idx, retaddr);
136         goto redo;
137     }
138     return res;
139 }

In this function, it will get the index of TLB table and compare the guest virtual address with the address stored in this tlb entry(line 97-100). If these two addresses match, it means this guest virtual address hits the tlb entry. Then qemu will determine this virtual address is a MMIO address or RAM address. If it is a MMIO address, get the index of IO emulation functions from env->iotlb and call these functions(line 103-117). If it is a RAM space, add the guest virtual address to addend to get the host virtual address(line 118-128). If there is no matched tlb entry, then fietch the entry from table l1_phys_map and insert the entry to tlb table(line 135).

5. an example

When fetching code from guest memory, the whole code path is as flowing:

cpu_exec->tb_find_fast->tb_find_slow->get_phys_addr_code->(if tlb not match)ldub_code(softmmu_header.h)->__ldl_mmu(softmmu_template.h)->tlb_fill->cpu_mips_handle_mmu_fault->tlb_set_page->tlb_set_page_exec

This article is based on qemu version 0.10.5 and target machine emulated is little endian mips. I will summarize the code path of mips lw instruction emulation in qemu.

Function decode_opc is used for decoding all the fetched instructions before tcg generating the target binary.

target-mips/translate.c

7566 static void decode_opc (CPUState *env, DisasContext *ctx)

7960     case OPC_LB ... OPC_LWR: /* Load and stores */
7961     case OPC_SB ... OPC_SW:
7962     case OPC_SWR:
7963     case OPC_LL:
7964     case OPC_SC:
7965          gen_ldst(ctx, op, rt, rs, imm);
7966          break;

It will call function gen_ldst which is also in target-mips/translate.c.

target-mips/translate.c

973 static void gen_ldst (DisasContext *ctx, uint32_t opc, int rt,
974                       int base, int16_t offset)

1046     case OPC_LW:
1047         op_ldst_lw(t0, ctx);
1048         gen_store_gpr(t0, rt);
1049         opn = "lw";
1050         break;

Function op_ldst_lw will generate the target binary which fetches the value from the emulated guest memory and gen_store_gpr will store this value to the emulated cpu’s general register rt.

Function op_ldst_lw is generated by the macro OP_LD.

target-mips/translate.c

901 #define OP_LD(insn,fname)                                        \
902 static inline void op_ldst_##insn(TCGv t0, DisasContext *ctx)    \
903 {                                                                \
904     tcg_gen_qemu_##fname(t0, t0, ctx->mem_idx);                  \
905 }

910 OP_LD(lw,ld32s);

We can find that op_ldst_lw is a function which calls function tcg_gen_qemu_ld32s. It will output the OPC(INDEX_op_qemu_ld32u) and args to gen_opc_ptr.

tcg/tcg-op.h

1793 static inline void tcg_gen_qemu_ld32s(TCGv ret, TCGv addr, int mem_index)
1794 {
1795 #if TARGET_LONG_BITS == 32
1796     tcg_gen_op3i_i32(INDEX_op_qemu_ld32u, ret, addr, mem_index);
1797 #else
1798     tcg_gen_op4i_i32(INDEX_op_qemu_ld32u, TCGV_LOW(ret), TCGV_LOW(addr),
1799                      TCGV_HIGH(addr), mem_index);
1800     tcg_gen_sari_i32(TCGV_HIGH(ret), TCGV_LOW(ret), 31);
1801 #endif
1802 }

99 static inline void tcg_gen_op3i_i32(int opc, TCGv_i32 arg1, TCGv_i32 arg2,
100                                     TCGArg arg3)
101 {
102     *gen_opc_ptr++ = opc;
103     *gen_opparam_ptr++ = GET_TCGV_I32(arg1);
104     *gen_opparam_ptr++ = GET_TCGV_I32(arg2);
105     *gen_opparam_ptr++ = arg3;
106 }

The path of generation of target binary code of tcg is as following.

cpu_gen_code->tcg_gen_code->tcg_gen_code_common->tcg_reg_alloc_op->tcg_out_op

tcg/i386/tcg-target.c

856 static inline void tcg_out_op(TCGContext *s, int opc,
857                               const TCGArg *args, const int *const_args)

1041     case INDEX_op_qemu_ld32u:
1042         tcg_out_qemu_ld(s, args, 2);
1043         break;

431 static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args,
432                             int opc)

508 #if TARGET_LONG_BITS == 32
509     tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_EDX, mem_index);
510 #else
511     tcg_out_mov(s, TCG_REG_EDX, addr_reg2);
512     tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_ECX, mem_index);
513 #endif
514     tcg_out8(s, 0xe8);
515     tcg_out32(s, (tcg_target_long)qemu_ld_helpers[s_bits] -
516               (tcg_target_long)s->code_ptr - 4);

In line 514, tcg outputs 0xe8 which means a call instruction in x86. It will call the functions in array qemu_ld_helpers. The args to the functions is passed by registers EAX,EDX and ECX.

tcg/i386/tcg-target.c

413 static void *qemu_ld_helpers[4] = {
414     __ldb_mmu,
415     __ldw_mmu,
416     __ldl_mmu,
417     __ldq_mmu,
418 };

These functions __ldb_mmu/__ldw_mmu are defined in softmmu_template.h.

softmmu_tempate.h

DATA_TYPE REGPARM glue(glue(__ld, SUFFIX), MMUSUFFIX)(target_ulong addr,
                                                      int mmu_idx)

In sum, function gen_ldst outputs the OPC(INDEX_op_qemu_ld32u) to gen_opc_ptr and tcg_out_op will generates the target binary according to the OPC. In the lw instruction emulation, it will generate the x86 binary calls the functions in softmmu_template.h.

Anyway, there are progress these days.

• Rewrite the GPIO I2C emulation for gdium. Now it is more clear than before.
• add st4180 rtc emulation to qemu
• add stds75 temperature sensor emulation to qemu
• change a little in uart emulation to satisfy pmon’s uart probing process
• fix a little bug in pflash_cfi02.c
• fix gdb stub bug in qemu to support mips64

Follow is the uart output of qemu-loongson.

kill-bill:/home/root/sdd/gdium/qemu-loongson/mips64el-softmmu# ./qemu-system-mips64el -M gdium -pflash gzrom.bin.gdb -nographic -S -s
Register sst39vf040  size 80000  at offset 08800000 addr 1fc00000 'pflash0' 80
devfn 70
unassigned_mem_readl Unassigned mem read 000000001fbffffc
unassigned_mem_readl Unassigned mem read 000000001fbffffc
new_sm502_mm_io 7000000 pci_mem _base 10000000
PMON2000 MIPS Initializing. Standby...
PRID=00006302
enable register space of MEMORY
DDR2 config begin_whd
DIMM read
0000008000000008read DIMM number of rows
read number of cols
module data width
DIMM SIZE=20000000
cols rows:
04030940DDR2 config end
DDR2 DLL locked
00000004
disable register space of MEMORY
jlliu : rom speed reg : 0x00000f8c
Init SDRAM Done!
Sizing caches...
Init caches...
godson2 caches found
Init caches done, cfg = 00030932
Copy PMON to execute location...
copy text section done.
Copy PMON to execute location done.
sp=80ffc000...............new_sm502_mm_io 6000000 pci_mem _base 10000000
cmd 7
mmio 6000000
FREQ
FREI
DONE
DEVI
ENVI
MAPV
nvram=bfc00000
NVRAM is invalid!
NVRAM@bfc00000
STDV
80100000: heap is already above this point
SBDD
P12PCIH

What I want to post here is not emulating I2C device directly, but emulating GPIO logic which pmon uses to emulate I2C. A little confused? Me too. Ok, let me talk the story from the began.

In pmon, it uses sm502 gpio pin 6 and 13 to emulate I2C master devices, which communicates with other slave device, such as SPD EEPROM and temperature sensors. I do not know why lemote uses gpio to emulate i2c, instead of using the I2C master in sm502. When the data is written to gpio pins according to I2C state machine, pmon expects the gpio pins acting like the I2C master devices. So qemu needs to emulate the I2C cycle accurate state machine, not only the I2C function.

I summarize the I2C emulation code logic of pmon here for reference.

(1) Start Bit

SCL=0 -> SDA=1 -> SCL=1 -> SDA=0 -> SCL=0

(2)Stop Bit

SCL=0 -> SDA=0 ->SCL=1 ->SDA=1 -> SCL=0

(3)Write a bit

Put data on sda -> SCL=1 -> SCL=0

(4)Write a byte

repeating (3) 8 times -> SDA=1

(5)Recv ACK

SCL=1 ->read sda until sda=0 -> SCL=0

(6)Read a bit

SCL=1 -> read data on SDA -> SCL=0

(7)Read a byte

repeating (6) 8 times

(8)Send ACK

put ACK on sda -> SCL=1 -> SCL=0

(9)Send a buffer

Do the following steps until the buffer is empty

1. START
2. Send slave address
3. Recv ACK
4. Send reg address
5. Recv ACK
6. Send one byte
7. Recv ACK
8. STOP

(10) Recv many bytes from slave deviceDo the following steps until finished

1. START
2. Send slave address
3. Recv ACK
4. Send reg address
5. Recv ACK
6. Repeat START
7. Send slave address+1(tell slave we want to recv)
8. Recv ACK
9. Recv a byte
10. Send ACK
11. STOP

For the sake of portability, the new version of qemu uses TCG, a tiny code generator, instead of dyngen to generate host code. That means one backend is needed for each host architecture. There are i386,x86_64,ppc,ppc64 and hppa backend in TCG, but MIPS is not on the list.

I confirm this conclusion on the status page of qemu web site. The MIPS's status is 'Not Supported'. Perhaps these days the most emergent work is to add MIPS host support, instead of loongson guest support to qemu.

One question: who is responsible for clear IP bits in cause register? That is interrupt handler. Interrupt handler must clear the corresponding IP bit in cause register, othewise next time, interrupt is reenabled, CPU thinks the interrupt is still happening!

In linux, the function to clear IP bit in mask_and_ack_intc_irq. In JZ4740, it is:

50 static void mask_and_ack_intc_irq(unsigned int irq)
51 {
52         __intc_mask_irq(irq);
53         __intc_ack_irq(irq);
54 }

1372 #define __intc_mask_irq(n)      ( REG_INTC_IMSR = (1 << (n)) )
1373 #define __intc_ack_irq(n)       ( REG_INTC_IPR = (1 << (n)) )

Linux writes to REG_INTC_IPR to clear pending interrupt and JZ4740 interrupt controller will clear the interrupt signal to MIPS CPU.

In qemu-jz, when writing to REG_INTC_IPR, the interrupt to MIPS cpu is not cleared and linux runs in a dead loop.

The following is the interrupt handling process in MIPS CPU.

1. Set ECP=PC and CPU jumps to 0x80000200 and then to handle_int. The interrupt is disabled.

2. Function handle_int will save all the register(including EPC so that interrupt can be nested) to stack and call plat_irq_dispatch.

3. After the irq is processed, it will reenable the in function __do_softirq. If a new interrupt is coming at this time(check the IP bits cause register), go to 1. otherwise go to 4.

4. Call ret_from_irq and restore all the register(including EPC) and using ERET to set PC=EPC.

5. Runs from PC.

Because of the IP bits in cause register is not cleared, linux will run into a dead loop between 1 and 3.

Thanks to the log function and remote gdb. Really hard to find this bug in qemu-jz.

2. Because twl4030 emulation is not complete, mmc device can not be found in linux kernel.

Next step:

1. twl4030 emulation. Big effort is needed.

2. dss/dma emulation

3. usb

<4>mtd->read(0x1f9f4 bytes from 0x2060c) returned ECC error

<4>mtd->read(0x1f9f4 bytes from 0x2060c) returned ECC error

<5>jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00020800: 0xb1e0 instead

jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00020800: 0xb1e0 instead

After hours of debuging, I find that the message is related to bugs of nand flash emulation.But I do not want to change nand.c code too much because I am afraid it will have some sideeffect to other type nand flash emulation. So I decide to wirte a new nand flash emulation for beagle board or other big page nand flash. After that, rootfs can be booted from nand flash.

However, MMC/USB/audio/LCD display/keyboard emulation are not ok . I will work on it.

In the following I will give you some informations of how to run qemu-omap3.

1. checkout qemu-omap3 from svn. Tag v0.01 is working.

svn checkout http://qemu-omap3.googlecode.com/svn/tags/v0.01 qemu-omap3

Or you can download it from http://qemu-omap3.googlecode.com/files/qemu-omap3-v0.01.tar.bz2 .

2. make qemu-omap3

cd qemu-omap3
./configure --target-list=arm-softmmu
make

qemu-system-arm will be generated in directory qemu-omap3/arm-softmmu.

3. download the x-loader/u-boot/kernel image.

cd qemu-omap3/arm-softmmu
wget http://qemu-omap3.googlecode.com/files/image-v0.01.tar.bz2
tar jxvf image-v0.01.tar.bz2

4. download the rootfs from beagle board site.

wget http://beagleboard.googlecode.com/files/rd-ext2-8M.bin

5. make nand flash image

cd qemu-omap3/arm-softmmu
cp ../bb_nandflash.sh .
cp ../bb_nandflash_ecc .
./bb_nandflash.sh x-load.bin.ift beagle-nand.bin x-loader
./bb_nandflash.sh u-boot.bin beagle-nand.bin u-boot
./bb_nandflash.sh uImage beagle-nand.bin kernel
./bb_nandflash.sh rd-ext2-8M.bin  beagle-nand.bin  rootfs
./bb_nandflash_ecc beagle-nand.bin 0x0 0xe80000

6. run qemu-omap3

./qemu-system-arm -M beagle -mtdblock beagle-nand.bin

7. During the graphical emulation, you can use the following keys:
Ctrl-Alt-f
Toggle full screen
Ctrl-Alt-n
Switch to virtual console 'n'. Standard console mappings are:
1
Target system display
2
Monitor
3
Serial port
Ctrl-Alt
Toggle mouse and keyboard grab.

Using Ctrl-Alt-3 to switch to beagle board serial port. You can see x-loader and u-boot booting message. When u-boot command line appears, type the following commands.

OMAP3 beagleboard.org # nand read 0x80000000 0x280000 0x400000
NAND read: device 0 offset 0x280000, size 0x400000
4194304 bytes read: OK
OMAP3 beagleboard.org # nand read 0x81600000 0x680000 0x800000
NAND read: device 0 offset 0x680000, size 0x800000
8388608 bytes read: OK
OMAP3 beagleboard.org # setenv bootargs 'console=ttyS2,115200n8 ramdisk_size=8192 root=/dev/ram0 rw rootfstype=ext2 initrd=0x81600000,8M nohz=0ff'
OMAP3 beagleboard.org # bootm 0x80000000

Linux will boot with some debug information of qemu-omap3. Enjoy it.

Update: 2010-02-20

I am too busy to do omap3 emulation these days. The guy at maemo has made great improvement in qemu omap3 emulation. Go to this page for more information. I do not know whether the steps in this artilcle can be used in the new version of qemu-omap3.

Linux kernel uses printk to output informations to console and the console struct is initialized in function console_init(). So why the linux can be displayed before function console_init()?

The answer is that there is another way to output informations to console. That is low level debug. It uses printascii to print informations. The following is the code in vprintk.

556 #ifdef  CONFIG_DEBUG_LL

557         printascii(printk_buf);

558 #endif

So if we want to output debug information as early as possible, define CONFIG_DEBUG_LL first.

The default configuration of beagle board in http://www.beagleboard.org/uploads/2.6_kernel_revb-v2.tar.gz defines this macro while the source code from  linux omap git does not define this macro.

Function printascii is defined in include/asm-arm/arch-omap/debug-macro.S.

OMAP3 beagleboard.org #
OMAP3 beagleboard.org # nand read 0x80000000 0x280000 0x400000
NAND read: device 0 offset 0x280000, size 0x400000
 4194304 bytes read: ERROR
OMAP3 beagleboard.org # bootm 0x80000000
## Booting kernel from Legacy Image at 80000000 ...
   Image Name:   Linux-2.6.22.18-omap3
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1856616 Bytes =  1.8 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... Bad Data CRC
ERROR: can't get kernel image!
OMAP3 beagleboard.org #

It means crc error in linux image. I use qemu console command 'memsave' to dump the data from address 0x80000000 to a file and compare it to uImage_OTG. The result is that linux image in ram(from 0x80000000) is different from uImage_OTG. In every block(256 bytes), there is one bit different from uImage_OTG. The linux image is changed when using 'nand read' command to read it from nand flash image to sram.

What has happened when reading linux image from nand flash image into sram??

After debuging about half a day, I find that the data are changed by the correct_data function. I *DID NOT* save ecc value in nand flash image so u-boot thinks the ecc value in oob is wrong and correct it. Ecc can only correct one bit in an block, that is why a bit has been changed in 256 bytes.

ecc_status = this->correct_data(mtd, &data_poi[datidx], &ecc_code[j], &ecc_calc[j]);

I write a file(bb_nandflash_ecc.c) to generate ecc value for nand flash image. Now I can boot linux image using u-boot command.

OMAP3 beagleboard.org # nand read 0x80000000 0x280000 0x400000
NAND read: device 0 offset 0x280000, size 0x400000
 4194304 bytes read: OK
OMAP3 beagleboard.org # bootm 0x80000000
## Booting kernel from Legacy Image at 80000000 ...
   Image Name:   Linux-2.6.22.18-omap3
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1856616 Bytes =  1.8 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK
Starting kernel ...
smc 80e80414  insn 1600070
Uncompressing Linux.......................................................................................................................... done, booting the kernel.
<5>Linux version 2.6.22.18-omap3 (root@fedoraserver) (gcc version 4.2.1 (CodeSourcery Sourcery G++ Lite 2007q3-51)) #1 Thu Jul 24 15:29:36 IST 2008
CPU: ARMv7 Processor [411fc082] revision 2 (ARMv7), cr=00003837
Machine: OMAP3 Beagle board
Memory policy: ECC disabled, Data cache writeback
<7>On node 0 totalpages: 32768
<7>  DMA zone: 256 pages used for memmap
<7>  DMA zone: 0 pages reserved
<7>  DMA zone: 32512 pages, LIFO batch:7
<7>  Normal zone: 0 pages used for memmap
Unassigned mem readl 4830a204

Because of omap3 emulation is not complete, linux kernel will stop after linux kernel booting. I will work on it.

Beagle board has an DVI transmitter TFP 410 and it outputs ditigal signal to LCD panel which has DVI connector. However, we do not care TFP 410 because it is just a pipe for LCD output data. What we care about is the LCD output data itself.

According to the datasheet of OMAP3530, it has three interfaces for LCD output:Parellel Interface, SDI interface and DSI interface. In the case of parellel interface, it can work in two modes: RFBI mode and RFBI bypass mode.

In RFBI mode, the datapath for LCD output data is display controller and RFBI chip. In RFBI bypass mode, the data is output to LCD panel or other interfaces directly. Beagle board uses TFP 410 to receive 24 bit LCD data and then transmits the data to LCD through DVI interface.

Qemu already has omap dss emulation. But it is not complete and only for RFBI mode(Nokia N800 uses RFBI). I just follow the Blizzard template and write my own omap3_lcd_panel_template. And then add the sdl update function omap3_lcd_panel_update_display to omap_dss.c. Also some other utility functions are needed. It works now, the beagle and penguin are displayed in my LCD panel. The following is the screenshot.

I have tested the LCD emulation speed and the result is not bad. It can reach about 20-30 frames per second. I think it is good enough for an emulator.

The next steps for qemu-omaps are: running linux kernel and distribution(Ångström for example). However the most exciting part is the 3D virtualization for OMAP3. That is you can run 3D application in qemu by using the host 3D accelerator hardware.

Please be patient. I am working on it.

]]> http://vm-kernel.org/blog/2008/12/04/lcd-is-working-in-qemu-omap3/feed/ 1 http://vm-kernel.org/blog/2008/11/28/qemu-omap-gp-timer-bug/ http://vm-kernel.org/blog/2008/11/28/qemu-omap-gp-timer-bug/#comments Fri, 28 Nov 2008 11:48:39 +0000 yajin http://vm-kernel.org/blog/2008/11/28/qemu-omap-gp-timer-bug/ 在u-boot中，采用函数udelay来进行延时。这是一个单纯等待的算法。在beagle board的u-boot中，udelay会不断去读OMAP general purpose timer的count值，然后根据general purpose timer的输入时钟来得到已经经过的时间。

在u-boot中，gp timer2的输入时钟为系统时钟，beagle board中为13MHZ。并且时钟经过了256分频。因此，每一秒钟的时间，gp timer2的count的值会增加1*(13MHZ/256)。

对于qemu来说，必须要保证当u-boot去读gptimer寄存器值的时候，返回的count值是正确的。也就是说，必须是实际经过的时间*13MHZ/256。
在qemu中，ticks_per_sec的值为1000000000。函数qemu_get_clock可以得到当前时间的ticks。因此，当u-boot来读gptimer寄存器值的时候，返回的count值应该为：

(qemu_get_clock/ticks_per_sec)*(13MHZ/256)

也就是

(qemu_get_clock*13MHZ)/(ticks_per_sec*256)

见函数omap_gp_timer_read

distance = muldiv64(distance, timer->rate, timer->ticks_per_sec);

这里，timer->ticks_per_sec= ticks_per_sec*256。

可是问题的关键在于，函数muldiv64的原型为

/* compute with 96 bit intermediate result: (a*b)/c */
uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c)

也就是说，参数b和c都是32bit的。而timer->ticks_per_sec已经超过了32bit。导致distance的计算出错，后果是u-boot中的延时函数是不准的。

解决方法：

首先判断一下ticks_per_secis有没有超过32bit，然后根据情况分别计算。

/*if ticks_per_secis bigger than 32bit, we can not use muldiv64 anymore!*/
if (timer->ticks_per_sec>0xffffffff){
  distance = distance/(ticks_per_sec/1000);  /*distance ms*/
  rate = timer->rate >> (timer->pre ? timer->ptv + 1 : 0);
  distance = muldiv64(distance, rate, 1000);
 }
else
  distance = muldiv64(distance, timer->rate, timer->ticks_per_sec);

现在U-boot中的时间是准确的了。

经过几天的debug，在很多地方加入 判断是否有SIG pending的语句，结果都是没有信号pending。那么唯一的可能就根本没有产生SIGALRM信号！
继续追踪。Qemu中提供了很多中定时器的实现方法，见数组alarm_timers。

1: static struct qemu_alarm_timer alarm_timers[] = {

2: #ifndef _WIN32

3: #ifdef __linux__

4:     {"dynticks", ALARM_FLAG_DYNTICKS, dynticks_start_timer,

5:      dynticks_stop_timer, dynticks_rearm_timer, NULL},

6:     /* HPET - if available - is preferred */

7:     {"hpet", 0, hpet_start_timer, hpet_stop_timer, NULL, NULL},

8:     /* ...otherwise try RTC */

9:     {"rtc", 0, rtc_start_timer, rtc_stop_timer, NULL, NULL},

10: #endif

11:     {"unix", 0, unix_start_timer, unix_stop_timer, NULL, NULL},

12: #else

13:     {"dynticks", ALARM_FLAG_DYNTICKS, win32_start_timer,

14:      win32_stop_timer, win32_rearm_timer, &alarm_win32_data},

15:     {"win32", 0, win32_start_timer,

16:      win32_stop_timer, NULL, &alarm_win32_data},

17: #endif

18:     {NULL, }

19: };

在初始化的时候，qemu会依次读取这个数组中所有的timer类型，只到某一个timer被初始化成功。在qemu-omap3中，dyntics会被初始化成功。也就是说，负责产生SIGALRM信号的就是dynticks。其启动timer和重新设置超时时间的是函数 dynticks_start_timer和dynticks_rearm_timer。

在dynticks_rearm_timer中会计算并设置超时时间，函数为qemu_next_deadline_dyntick。问题就在 qemu_next_deadline_dyntick中。

这个函数一开始计算virtualtime的超时时间，然后在存在 realtime的timer的时候，才计算realtime的超时时间，然后取两者最小值。关键在于当没有realtime的timer的时候，得到的 virtualtimer值过大，然后设置了这个很多的值作为下一次的超时时间。导致很长时间都不会出现超时信号。现在在函数 qemu_next_deadline_dyntick中增加了一个最大值的限制。

1: if (delta>MAX_TIMER_REARM_US)

2:  delta = MAX_TIMER_REARM_US;

问题解决了。现在qemu-omap3可以接收键盘输入了。

而在qemu中的定时器是通过函数host_alarm_handler来触发运行的，函数host_alarm_handler又是通过信号来进行触发的。那么问题的根源就是信号没有被发送或者被block(还没有查出具体原因)。

我尝试在终端中向qemu发生SIGALRM信号，果然，qemu接收到SIGALRM信号后，执行host_alarm_handler然后会执行所有注册了的定时器的超时函数(包括display_state.gui_timer)。

kill-bill: # ps aux | grep qemu

root      2945  101  3.2 196468  8404 pts/1    R+   09:36   0:14 ./qemu-system-arm -M beagle -mtdblock beagle-nand.bin

root      2947  0.0  0.2   2996   692 pts/0    R+   09:36   0:00 grep qemu

kill-bill: # kill -s SIGALRM 2945

多次发送SIGALRM信号后，qemu可以正常接受终端的输入。下面是qemu运行u-boot的截图。

接下来就需要查找为什么qemu没有收到SIGALRM！

1. 对判断外部输入时钟频率的部分，做了修改。x-load判断外部时钟频率的方法是先设置32KHZ的基准时钟，然后设置GP1的时钟源为SYS_CLK。然后读取GP1 Timer在20个cycle内的差值，进而判断时钟。这个方法在emulator中有的时候并不是很准。因为emulator除了执行指令外，还需要做其他事情。有的时候，emulator去读取GP1 Timer 的时候，实际上运行的时间已经不止ARM 20个cycle的时间，因此读取出来的counter值偏大。而x-load仍然认为刚才是跑了20个cycle。因此，得到的频率比实际的频率大。在 qemu-omap3中，目前认为外部时钟的频率是12MHZ13MHZ,因此，去掉了x-load中判断外部时钟频率的部分，在x-load中直接赋值外部时钟频率12MHZ13MHZ.

2. 去掉了nand flash中进入ECC检查的部分。也就是注释掉drivers/k9f1g08r0a.c中的宏ECC_CKECK_ENABLE

// #define ECC_CHECK_ENABLE

下面是x-load运行的信息：

# ./qemu-system-arm -M beagle -mtdblock beagle-nand.bin -serial stdio -nographic

Texas Instruments X-Loader 1.41

Starting OS Bootloader...

运行到u-boot的时候还有点问题。go on working.

Fabrice 的回复：http://lists.gnu.org/archive/html/qemu-devel/2006-11/msg00149.html

这里还有qemu 中3D加速的讨论：http://www.mail-archive.com/qemu-devel@nongnu.org/msg15702.html