Multi Threading

Multi Threading

This page documents all kernel functionality for managing multiple processes and threads as well as handling synchronization between them.

Processes #

Each process is given an array of kernel capability descriptors upon creation (see CreateProcess). Official software forwards the descriptors specified in the NCCH exheader.

Any process can only use SVCs which are enabled in its kernel capability descriptors. This is enforced by the ARM11 kernel SVC handler by checking the syscall access control mask stored on the SVC-mode stack. If the SVC isn’t enabled, a kernelpanic() is triggered. Each process has a separate SVC-mode stack; this stack and the syscall access mask stored here are initialized when the process is started. Applications normally only have access to SVCs <=0x3D, however not all SVCs <=0x3D are accessible to the application. The majority of the SVCs accessible to applications are unused by the application.

Each process has a separate handle-table, the size of which is stored in the kernel capability descriptor. The handles in a handle-table can’t be used in the context of other processes, since those handles don’t exist in other handle-tables.

0xFFFF8001 is a handle alias for the current process.

Calling svcBreak on retail will only terminate the process which called this SVC.

Usage #

CreateCodeSet #

(behavior unconfirmed)

Allocates memory for a process according to the given CodeSetInfo contents and copies the segment data from the given memory locations to the allocated memory.

CreateProcess #

(behavior unconfirmed)

Sets up a process using the segments managed by the given CodeSet handle.

This system call furthermore processes the kernel capabilities from the ExHeader, hence setting up virtual address mappings, CPU clock frequency/L2 cache configuration, and other things.

Run #

(behavior unconfirmed)

Sets up the main process thread and appends it to the scheduler queue.

The argc, argv, and envp fields from the given StartupInfo structure are ignored.

struct CodeSetInfo #

All addresses are given virtual for the process to be created. All sizes are given in 0x1000-pages.

TypeField
u8[8]Codeset Name
u16Unknown, this is written to field 0x5A of KCodeSet
u16Unknown/padding
u32Unknown/padding
u32.text addr
u32.text size
u32.rodata start
u32.rodata size
u32RW addr (.data + .bss)
u32RW size (.data + .bss)
u32Total .text pages
u32Total .rodata pages
u32Total RW pages (.data + .bss)
u32Unknown/padding
u8[8]Program ID

Threads #

For Kernel implementation details, see KThread.

Though it is possible to run multi-threaded programs, running those on different cores is not possible “as-is”. One core is always dedicated to the OS, hence you will never get 100% of both cores.

Using CloseHandle() with a KThread handle will terminate the specified thread only if the reference count reaches 0.

Lower priority values give the thread higher priority. For userland apps, priorities between 0x18 and 0x3F are allowed. The priority of the app’s main thread seems to be 0x30.

The appcore thread scheduler, in typical real-time operating system fashion, implements a simple preemptive algorithm based around multiple thread priority levels. This algorithm guarantees that the currently executing thread is always the highest priority runnable thread (also known as SCHED_FIFO). In other words, a thread will be interrupted (preempted) if and only if a higher priority thread is woken up, by means of an event (i.e. svcSendSyncRequest) or similar. Contrary to typical desktop operating systems, no timeslice-based scheduling is performed, which means that if a thread uses up all available CPU time (for example if it enters an endless loop), all other threads with equal or lower priority that run on the same CPU core won’t get a chance to run. Yielding to other threads is otherwise done by means of synchronization primitives (thread sleep, mutex, address arbiter, etc.). Address arbiters can be used to implement process-local synchronization primitives.

0xFFFF8000 is a handle alias for the currently active thread.

Usage #

CreateThread #

svc : 0x08

Signature

Result CreateThread(Handle* thread, func entrypoint, u32 arg, u32 stacktop, s32 threadpriority, s32 processorid);

Configuration

R0=s32 threadpriority
R1=func entrypoint
R2=u32 arg
R3=u32 stacktop
R4=s32 processorid
Result result=R0
Handle* thread=R1

Details

Creates a new thread in the current process which will begin execution at the given entrypoint. The SP CPU register will be initialized to stacktop, while r0 will be initialized to the given arg.

The input address used for Entrypoint_Param and StackTop are normally the same, but they may be chosen arbitrarily. For the main thread (created in svcRun), the Entrypoint_Param is value 0.

The stacktop must be aligned to 0x8-bytes, otherwise when not aligned to 0x8-bytes the ARM11 kernel clears the low 3-bits of the stacktop address.

The processorid parameter specifies which processor the thread can run on. Non-negative values correspond to a specific CPU. (e.g. 0 for the Appcore and 1 for the Syscore on Old3DS. On New3DS, IDs 2 and 3 are also valid, referring to the 2 additional CPU cores) Special value -1 means all CPUs, and -2 means the default CPU for the process (Read from the Exheader, usually 0 for applications, 1 for system services). Games usually create threads using -2.

The thread priority value must be in the range 0x0..0x3F. Otherwise, error 0xE0E01BFD is returned.

With the Old3DS kernel, the s32 processorid must be <=2 (for the processorid validation check in the kernel). With the New3DS kernel, the processorid validation check requires processorid to be less than or equal to <total cores(MPCore “SCU Configuration Register” CPU number value + 1)>, and a number of additional constraints apply: When processorid==0x2 and the process is not a BASE mem-region process, exheader kernel-flags bitmask 0x2000 must be set (otherwise error 0xD9001BEA is returned). When processorid==0x3 and the process is not a BASE mem-region process, error 0xD9001BEA is returned. These are the only restriction checks done by the kernel for processorid.

ExitThread #

svc : 0x09

Signature

void ExitThread(void);

Details

Makes the currently running thread exit. When a thread exits, all mutex objects it owns are released and made available to other threads.

SleepThread #

svc : 0x0A

Signature

void SleepThread(s64 nanoseconds);

GetThreadPriority #

svc : 0x0B

Signature

Result GetThreadPriority(s32* priority, Handle thread);

asm

.global svcGetThreadPriority
.type svcGetThreadPriority, %function
svcGetThreadPriority:
   str r0, [sp, #-0x4]!
   svc 0x0B
   ldr r3, [sp], #4
   str r1, [r3]
   bx  lr

SetThreadPriority #

svc : 0x0C

Signature

Result SetThreadPriority(Handle thread, s32 priority);

OpenThread #

svc : 0x34

Signature

Result OpenThread(Handle* thread, Handle process, u32 threadId);

GetProcessIdOfThread #

svc : 0x36

Signature

Result GetProcessIdOfThread(u32* processId, Handle thread);

GetThreadId #

svc : 0x37

Signature

Result GetThreadId(u32* threadId, Handle thread);

GetThreadInfo #

svc : 0x2C

Signature

Result GetThreadInfo(s64* out, Handle thread, ThreadInfoType type);

Details This requests always return an error when called, it only checks if the handle is a thread or not. Hence, it will return 0xD8E007ED (BAD_ENUM) if the Handle is a Thread Handle, 0xD8E007F7 (BAD_HANDLE) if it isn’t.

GetThreadContext #

svc : 0x3B

Signature

Result GetThreadContext(ThreadContext* context, Handle thread);

Details Stubbed?

Core affinity #

The cores are numbered from 0 to 1 for Old 3DS and 0 to 3 for the new 3DS.

GetThreadAffinityMask #

svc : 0x0D

Signature

Result GetThreadAffinityMask(u8* affinitymask, Handle thread, s32 processorcount);

SetThreadAffinityMask #

svc : 0x0E

Signature

Result SetThreadAffinityMask(Handle thread, u8* affinitymask, s32 processorcount);

GetThreadIdealProcessor #

svc : 0x0F

Signature

Result GetThreadIdealProcessor(s32* processorid, Handle thread);

SetThreadIdealProcessor #

svc : 0x10

APT:SetApplicationCpuTimeLimit #

See APT:SetApplicationCpuTimeLimit.

You are not able to use the system core (core1) by default. You have to first assign the amount of time dedicated to the system. The value is in percent, the higher it is, the more the system will be available for your application.

For example if you set this value to 25%, it means that your application will be able to use 25% of the system core at most, even if you never issue system calls.

If you set the value to a non-zero value, you will not be able to set it back to 0%. Keep in mind that if your application is heavily dependant on the system, setting a high value for your application might yield poorer performance than if you had set a low value.

APT:GetApplicationCpuTimeLimit #

See APT:GetApplicationCpuTimeLimit.

Debug #

GetThreadList #

GetDebugThreadContext #

SetDebugThreadContext #

GetDebugThreadParam #

Synchronization #

Synchronization can be performed via WaitSynchronization on any handles deriving from KSynchronizationObject. The semantic meaning of the call depends on the particular object type referred to by the given handle:

  • KClientPort: Wakes if max sessions not reached (free session available)
  • KClientSession: Always false?
  • KDebug: Waits until a debug event is signaled (the user should then use svcGetProcessDebugEvent to get the debug event info)
  • KDmaObject: ???
  • KEvent: Waits until the event is signaled
  • KMutex: Acquires a lock on the mutex (blocks until this succeeds)
  • KProcess: Waits until the process exits/is terminated
  • KSemaphore: This consumes a value from the semaphore count, if possible, otherwise continues to wait
  • KServerPort: Waits for a new client connection, upon which svcAcceptSession is ready to be called
  • KServerSession: Waits for an IPC command to be submitted to the server process
  • KThread: Waits until the thread terminates
  • KTimer: Wakes when timer activates (this also clears the timer if it is oneshot)

Most synchronization systems seem to have both a “normal” and “light-weight” version

Mutex #

For Kernel implementation details, see KMutex

CreateMutex #

/!\ It seems that the mutex will not be available once the thread that created it is destroyed

ReleaseMutex #

Semaphore #

Event #

Address Arbiters #

Address arbiters are a low-level primitive to implement synchronization based on a counter stored at some user-specified virtual memory address. Address arbiters are used to put the current thread to sleep until the counter is signaled. Both of these tasks are implemented in ArbitrateAddress.

Address arbiters are implemented by KAddressArbiter.

CreateAddressArbiter #

Result CreateAddressArbiter(Handle* arbiter)

Creates an address arbiter handle for use with ArbitrateAddress.

ArbitrateAddress #

Result ArbitrateAddress(Handle arbiter, u32 addr, ArbitrationType type, s32 value, s64 nanoseconds)

if type is SIGNAL, the ArbitrateAddress call will resume up to value of the threads waiting on addr using an arbiter, starting with the highest-priority threads. If value is negative, all of these threads are released. nanoseconds remains unused in this mode.

The other modes are used to (conditionally) put the current thread to sleep based on the memory word at virtual address addr until another thread signals that address using ArbitrateAddress with the type SIGNAL. WAIT_IF_LESS_THAN will put the current thread to sleep if that word is smaller than value. DECREMENT_AND_WAIT_IF_LESS_THAN will furthermore decrement the memory value before the comparison. WAIT_IF_LESS_THAN_TIMEOUT and DECREMENT_AND_WAIT_IF_LESS_THAN_TIMEOUT will do the same as their counterparts, but will have thread execution resume if nanoseconds nanoseconds pass without addr being signaled.

enum ArbitrationType #

Address arbitration typeValue
SIGNAL0
WAIT_IF_LESS_THAN1
DECREMENT_AND_WAIT_IF_LESS_THAN2
WAIT_IF_LESS_THAN_TIMEOUT3
DECREMENT_AND_WAIT_IF_LESS_THAN_TIMEOUT4