GCD Source Analysis 6 - dispatch_source

Preface

Compared with other parts of GCD, dispatch_source is probably the least eye-catching one. It is a wrapper around kqueue, a common BSD kernel facility, and can monitor certain kinds of events. When those events occur, it automatically places a block onto a dispatch queue for execution. If you already know a little about I/O multiplexing, dispatch_source is much easier to understand.

kqueue

kqueue is one BSD implementation of I/O multiplexing. Its interface mainly consists of two system calls, kqueue() and kevent(), plus the struct kevent structure.

    1. kqueue() creates a kernel event queue and returns its file descriptor.
1
int     kqueue(void);
    1. kevent() registers or unregisters events with the kernel, and returns ready events or error events.
1
2
3
4
int     kevent(int kq, 
const struct kevent *changelist, int nchanges,
struct kevent *eventlist, int nevents,
const struct timespec *timeout);
    1. struct kevent is the basic event structure operated on by kevent().
1
2
3
4
5
6
7
8
struct kevent { 
uintptr_t ident; /* event ID */
short filter; /* event filter */
u_short flags; /* action flags */
u_int fflags; /* filter-specific flags */
intptr_t data; /* filter data */
void *udata; /* user data passed through by the app */
};

Inside one kqueue, {ident, filter} uniquely identifies an event:

    1. ident is the event ID. It is usually set to a file descriptor.
    1. filter can be understood as the event type. The kernel checks the state of the filter registered on ident, and notifies the application when that state changes. kqueue defines many filters:
1
2
3
4
5
6
7
8
9
10
#define EVFILT_READ         (-1)
#define EVFILT_WRITE (-2)
#define EVFILT_AIO (-3) /* attached to aio requests */
#define EVFILT_VNODE (-4) /* attached to vnodes */
#define EVFILT_PROC (-5) /* attached to struct proc */
#define EVFILT_SIGNAL (-6) /* attached to struct proc */
#define EVFILT_TIMER (-7) /* timers */
#define EVFILT_MACHPORT (-8) /* Mach portsets */
#define EVFILT_FS (-9) /* Filesystem events */
#define EVFILT_USER (-10) /* User events */
    1. The behavior flags:
1
2
3
4
#define EV_ADD              0x0001      /* add event to kq (implies enable) */
#define EV_DELETE 0x0002 /* delete event from kq */
#define EV_ENABLE 0x0004 /* enable event */
#define EV_DISABLE 0x0008 /* disable event (not reported) */

Event Types

In source.h, dispatch_source supports the following event types:

1
2
3
4
5
6
7
8
9
10
11
DISPATCH_SOURCE_TYPE_DATA_ADD:       custom event
DISPATCH_SOURCE_TYPE_DATA_OR: custom event
DISPATCH_SOURCE_TYPE_MACH_SEND: Mach port send event
DISPATCH_SOURCE_TYPE_MACH_RECV: Mach port receive event
DISPATCH_SOURCE_TYPE_PROC: process-related event
DISPATCH_SOURCE_TYPE_READ: file read event
DISPATCH_SOURCE_TYPE_WRITE: file write event
DISPATCH_SOURCE_TYPE_VNODE: file attribute change event
DISPATCH_SOURCE_TYPE_SIGNAL: signal event
DISPATCH_SOURCE_TYPE_TIMER: timer event
DISPATCH_SOURCE_TYPE_MEMORYPRESSURE: memory pressure event

Usage Steps

Because dispatch_source is used less often, it is worth introducing its basic usage:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// 1. Create a dispatch source. Here data is coalesced by addition.
// The last parameter specifies the dispatch queue.
dispatch_source_t source = dispatch_source_create(dispatch_source_type, handler, mask, dispatch_queue);

// 2. Set the block that responds to dispatch source events.
// The block runs on the queue specified by the dispatch source.
dispatch_source_set_event_handler(source, ^{
// dispatch_source_get_data(source) can be used to get source data.
});

// 3. A dispatch source is suspended after creation, so it must be resumed.
dispatch_resume(source);

// 4. Merge data into the dispatch source.
dispatch_source_merge_data(source, value);

Timer

When using timers, NSTimer is usually the first thing that comes to mind. But NSTimer is affected by the RunLoop. When the RunLoop has many tasks to process, timer precision can drop. For cases that need very high timing precision, we may consider CADisplayLink; in fact, GCD timers are also an option.

1
2
3
4
5
6
7
8
9
dispatch_queue_t queue = dispatch_get_global_queue(0, 0);
dispatch_queue_t timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, queue);

dispatch_source_set_timer(self.timer, dispatch_time(DISPATCH_TIME_NOW, 3.0 * NSEC_PER_SEC), 2.0 * NSEC_PER_SEC, 0);

dispatch_source_set_event_handler(timer, ^{
// handling logic
});
dispatch_resume(self.timer);

Monitoring Files

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
dispatch_source_t fileMonitor(const char* filename) 
{
int fd = open(filename, O_EVTONLY);
if (fd == -1) return NULL;

dispatch_queue_t queue = dispatch_get_global_queue(0, 0);
dispatch_source_t source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE, fd, DISPATCH_VNODE_RENAME, queue);

int length = strlen(filename);
char* newString = (char*)malloc(length + 1);
newString = strcpy(newString, filename);
dispatch_set_context(source, newString);

dispatch_source_set_event_handler(source, ^{
const char* oldFilename = (char*)dispatch_get_context(source);
...
});

dispatch_source_set_cancel_handler(source, ^{
char* fileStr = (char*)dispatch_get_context(source);
free(fileStr);
close(fd);
});

dispatch_resume(source);

return source;
}

Common API Source Analysis

dispatch_source_s

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
struct dispatch_source_s {
DISPATCH_STRUCT_HEADER(dispatch_source_s, dispatch_source_vtable_s);
DISPATCH_QUEUE_HEADER;
union {
char _ds_pad[DISPATCH_QUEUE_MIN_LABEL_SIZE];
struct {
char dq_label[8];
dispatch_kevent_t ds_dkev;
dispatch_source_handler_function_t ds_handler_func;
void *ds_handler_ctxt;
void *ds_cancel_handler;
unsigned int ds_is_level:1,
ds_is_adder:1,
ds_is_installed:1,
ds_needs_rearm:1,
ds_is_armed:1,
ds_is_legacy:1,
ds_cancel_is_block:1,
ds_handler_is_block:1;
unsigned int ds_atomic_flags;
unsigned long ds_data;
unsigned long ds_pending_data;
unsigned long ds_pending_data_mask;
TAILQ_ENTRY(dispatch_source_s) ds_list;
unsigned long ds_ident_hack;
struct dispatch_timer_source_s ds_timer;
};
};
};

If you still remember the definition of dispatch_queue_s, you can see that dispatch_source_s is almost an overlay version of dispatch_queue_s:

1
2
3
4
5
struct dispatch_queue_s {
DISPATCH_STRUCT_HEADER(dispatch_queue_s, dispatch_queue_vtable_s);
DISPATCH_QUEUE_HEADER;
char dq_label[DISPATCH_QUEUE_MIN_LABEL_SIZE]; // must be last
};

Because the third field of dispatch_source_s is a union, char _ds_pad[DISPATCH_QUEUE_MIN_LABEL_SIZE]; and char dq_label[DISPATCH_QUEUE_MIN_LABEL_SIZE]; in dispatch_queue_s are basically the same thing. This lets functions for dispatch_queue_s initialize dispatch_source_s, and then source-specific fields are overwritten afterward.

dispatch_source_create

As usual, start from creation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
dispatch_source_t 
dispatch_source_create(dispatch_source_type_t type,
uintptr_t handle,
unsigned long mask,
dispatch_queue_t q)
{
const struct kevent *proto_kev = &type->ke;

dispatch_source_t ds = NULL;
dispatch_kevent_t dk = NULL;

...

ds = calloc(1ul, sizeof(struct dispatch_source_s));
dk = calloc(1ul, sizeof(struct dispatch_kevent_s));

dk->dk_kevent = *proto_kev;
dk->dk_kevent.ident = handle;
dk->dk_kevent.flags |= EV_ADD|EV_ENABLE;
dk->dk_kevent.fflags |= (uint32_t)mask;
dk->dk_kevent.udata = dk;
TAILQ_INIT(&dk->dk_sources);

_dispatch_queue_init((dispatch_queue_t)ds);
strlcpy(ds->dq_label, "source", sizeof(ds->dq_label));

ds->do_vtable = &_dispatch_source_kevent_vtable;
ds->do_ref_cnt++;
ds->do_suspend_cnt = DISPATCH_OBJECT_SUSPEND_INTERVAL;
ds->do_targetq = q;

ds->ds_ident_hack = dk->dk_kevent.ident;
ds->ds_dkev = dk;
ds->ds_pending_data_mask = dk->dk_kevent.fflags;
if ((EV_DISPATCH|EV_ONESHOT) & proto_kev->flags) {
if (proto_kev->filter != EVFILT_MACHPORT) {
ds->ds_is_level = true;
}
ds->ds_needs_rearm = true;
} else if (!(EV_CLEAR & proto_kev->flags)) {
ds->ds_is_adder = true;
}

if (type->ke.filter == DISPATCH_EVFILT_TIMER) {
ds->ds_needs_rearm = true;
}

if (type == DISPATCH_SOURCE_TYPE_MACH_SEND) {
static dispatch_once_t pred;
dispatch_once_f(&pred, NULL, _dispatch_mach_notify_source_init);
} else if (type == DISPATCH_SOURCE_TYPE_TIMER) {
ds->ds_timer.flags = mask;
}

_dispatch_retain(ds->do_targetq);
return ds;

out_bad:
free(ds);
free(dk);
return NULL;
}

Two points are worth noting:

    1. dispatch_source_type_s has two fields: one kevent and one mask.
1
2
3
4
5
6
7
8
9
10
11
struct dispatch_source_type_s {
struct kevent ke;
uint64_t mask;
};

const struct dispatch_source_type_s _dispatch_source_type_timer = {
.ke = {
.filter = DISPATCH_EVFILT_TIMER,
},
.mask = DISPATCH_TIMER_INTERVAL|DISPATCH_TIMER_ONESHOT|DISPATCH_TIMER_ABSOLUTE|DISPATCH_TIMER_WALL_CLOCK,
};
    1. do_vtable points to the source-specific vtable.
1
2
3
4
5
6
7
8
const struct dispatch_source_vtable_s _dispatch_source_kevent_vtable = {
.do_type = DISPATCH_SOURCE_KEVENT_TYPE,
.do_kind = "kevent-source",
.do_invoke = _dispatch_source_invoke,
.do_dispose = _dispatch_source_dispose,
.do_probe = _dispatch_source_probe,
.do_debug = _dispatch_source_kevent_debug,
};

dispatch_source_set_event_handler

1
2
3
4
5
6
void dispatch_source_set_event_handler(dispatch_source_t ds, dispatch_block_t handler)
{
handler = _dispatch_Block_copy(handler);
dispatch_barrier_async_f((dispatch_queue_t)ds,
handler, _dispatch_source_set_event_handler2);
}

First, the handler is copied. Then dispatch_barrier_async_f is called. This method is extremely similar to dispatch_async_f from the queue article: it sets the block on a continuation and pushes it into the target queue. Also notice that dispatch_barrier_async_f treats ds directly as a queue. That is reasonable because their structure definitions are highly similar.

1
2
3
4
5
6
7
8
9
10
11
12
void dispatch_barrier_async_f(dispatch_queue_t dq, void *context, dispatch_function_t func)
{
dispatch_continuation_t dc = fastpath(_dispatch_continuation_alloc_cacheonly());

if (!dc) return _dispatch_barrier_async_f_slow(dq, context, func);

dc->do_vtable = (void *)(DISPATCH_OBJ_ASYNC_BIT | DISPATCH_OBJ_BARRIER_BIT);
dc->dc_func = func;
dc->dc_ctxt = context;

_dispatch_queue_push(dq, dc);
}

Now look at _dispatch_source_set_event_handler2:

1
2
3
4
5
6
7
8
9
10
11
12
13
static void _dispatch_source_set_event_handler2(void *context)
{
struct Block_layout *bl = context;

dispatch_source_t ds = (dispatch_source_t)_dispatch_queue_get_current();

if (ds->ds_handler_is_block && ds->ds_handler_ctxt) {
Block_release(ds->ds_handler_ctxt);
}
ds->ds_handler_func = bl ? (void *)bl->invoke : NULL;
ds->ds_handler_ctxt = bl;
ds->ds_handler_is_block = true;
}

This method mainly stores the context for later use.

dispatch_source_set_cancel_handler

1
2
3
4
5
6
7
void dispatch_source_set_cancel_handler(dispatch_source_t ds,
dispatch_block_t handler)
{
handler = _dispatch_Block_copy(handler);
dispatch_barrier_async_f((dispatch_queue_t)ds,
handler, _dispatch_source_set_cancel_handler2);
}

This sets the cancel callback, showing that sources support cancellation.

1
2
3
4
5
6
7
8
9
10
11
static void _dispatch_source_set_cancel_handler2(void *context)
{
dispatch_source_t ds = (dispatch_source_t)_dispatch_queue_get_current();
dispatch_assert(ds->do_vtable == &_dispatch_source_kevent_vtable);

if (ds->ds_cancel_is_block && ds->ds_cancel_handler) {
Block_release(ds->ds_cancel_handler);
}
ds->ds_cancel_handler = context;
ds->ds_cancel_is_block = true;
}

dispatch_source_cancel

1
2
3
4
5
6
7
8
9
10
void dispatch_source_cancel(dispatch_source_t ds)
{
_dispatch_retain(ds);

dispatch_atomic_or(&ds->ds_atomic_flags, DSF_CANCELED);

_dispatch_wakeup(ds);

_dispatch_release(ds);
}

It sets the cancellation flag, wakes the source, performs cancellation, unregisters the source, and finally releases resources.

dispatch_source_get_xxx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
unsigned long dispatch_source_get_mask(dispatch_source_t ds)
{
return ds->ds_pending_data_mask;
}

uintptr_t dispatch_source_get_handle(dispatch_source_t ds)
{
return (int)ds->ds_ident_hack;
}

unsigned long dispatch_source_get_data(dispatch_source_t ds)
{
return ds->ds_data;
}

dispatch_source_merge_data

dispatch_source_merge_data merges data into a dispatch source of type DISPATCH_SOURCE_TYPE_DATA_ADD or DISPATCH_SOURCE_TYPE_DATA_OR, and submits its event handler block to the target queue.

1
2
3
4
5
6
7
8
9
void dispatch_source_merge_data(dispatch_source_t ds, unsigned long val)
{
struct kevent kev = {
.fflags = (typeof(kev.fflags))val,
.data = val,
};

_dispatch_source_merge_kevent(ds, &kev);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
void _dispatch_source_merge_kevent(dispatch_source_t ds, const struct kevent *ke)
{
struct kevent fake;

if ((ds->ds_atomic_flags & DSF_CANCELED) || (ds->do_xref_cnt == 0)) {
return;
}

if (ke->flags & EV_ERROR) {
if (ke->filter == EVFILT_PROC && ke->data == ESRCH) {
fake = *ke;
fake.flags &= ~EV_ERROR;
fake.fflags = NOTE_EXIT;
fake.data = 0;
ke = &fake;
} else {
dispatch_assume_zero(ke->data);
return;
}
}

if (ds->ds_is_level) {
dispatch_assert(ke->data >= 0l);
ds->ds_pending_data = ~ke->data;
}
else if (ds->ds_is_adder) {
dispatch_atomic_add(&ds->ds_pending_data, ke->data);
}
else {
dispatch_atomic_or(&ds->ds_pending_data, ke->fflags & ds->ds_pending_data_mask);
}

if (ds->ds_needs_rearm) {
ds->ds_is_armed = false;
}

_dispatch_wakeup(ds);
}

dispatch_resume

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
void dispatch_resume(dispatch_object_t dou)
{
if (slowpath(dou._do->do_ref_cnt == DISPATCH_OBJECT_GLOBAL_REFCNT)) {
return;
}
switch (dispatch_atomic_sub(&dou._do->do_suspend_cnt, DISPATCH_OBJECT_SUSPEND_INTERVAL) + DISPATCH_OBJECT_SUSPEND_INTERVAL) {
case DISPATCH_OBJECT_SUSPEND_INTERVAL:
_dispatch_wakeup(dou._do);
break;
case 0:
DISPATCH_CLIENT_CRASH("Over-resume of an object");
break;
default:
break;
}
}

From the creation process above, a newly created source normally has do_suspend_cnt set to DISPATCH_OBJECT_SUSPEND_INTERVAL, meaning it starts in a suspended state and must be manually started with dispatch_resume.

The switch branch above should obviously go through DISPATCH_OBJECT_SUSPEND_INTERVAL, which calls _dispatch_wakeup.

_dispatch_wakeup was also analyzed in the queue article. It is mainly related to do_invoke. For a source, do_invoke points to _dispatch_source_invoke. This method executes all source actions. Each action is guaranteed to run on the proper queue. If the current queue does not match the action, the correct queue is returned, and the action later runs on that returned queue. This is the core focus of this article.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
dispatch_queue_t _dispatch_source_invoke(dispatch_source_t ds)
{
dispatch_queue_t dq = _dispatch_queue_get_current();

if (!ds->ds_is_installed) {
if (dq != &_dispatch_mgr_q) {
return &_dispatch_mgr_q;
}

_dispatch_kevent_merge(ds);
}
else if ((ds->ds_atomic_flags & DSF_CANCELED) || (ds->do_xref_cnt == 0)) {
if (ds->ds_dkev) {
if (dq != &_dispatch_mgr_q) {
return &_dispatch_mgr_q;
}
_dispatch_kevent_release(ds);
return ds->do_targetq;
}
else if (ds->ds_cancel_handler) {
if (dq != ds->do_targetq) {
return ds->do_targetq;
}
}
_dispatch_source_cancel_callout(ds);
}
else if (ds->ds_pending_data) {
if (dq != ds->do_targetq) {
return ds->do_targetq;
}
_dispatch_source_latch_and_call(ds);

if (ds->ds_needs_rearm) {
return &_dispatch_mgr_q;
}
}
else if (ds->ds_needs_rearm && !ds->ds_is_armed) {
if (dq != &_dispatch_mgr_q) {
return &_dispatch_mgr_q;
}
_dispatch_kevent_resume(ds->ds_dkev, 0, 0);
ds->ds_is_armed = true;
}

return NULL;
}

This function mentions the manager queue, _dispatch_mgr_q, many times. Some cases must switch to the manager queue, for example:

  • ds_is_installed is false.
  • ds_dkev is not empty.
  • ds_needs_rearm is true.

The manager queue is analyzed later.

_dispatch_kevent_merge

  • Find existing kevents, and merge any new flags if necessary.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
void _dispatch_kevent_merge(dispatch_source_t ds)
{
static dispatch_once_t pred;
dispatch_kevent_t dk;
typeof(dk->dk_kevent.fflags) new_flags;
bool do_resume = false;

if (ds->ds_is_installed) {
return;
}
ds->ds_is_installed = true;

dispatch_once_f(&pred, NULL, _dispatch_source_init_tail_queue_array);

dk = _dispatch_kevent_find(ds->ds_dkev->dk_kevent.ident, ds->ds_dkev->dk_kevent.filter);

if (dk) {
new_flags = ~dk->dk_kevent.fflags & ds->ds_dkev->dk_kevent.fflags;
dk->dk_kevent.fflags |= ds->ds_dkev->dk_kevent.fflags;
free(ds->ds_dkev);
ds->ds_dkev = dk;
do_resume = new_flags;
}
else {
dk = ds->ds_dkev;
_dispatch_kevent_insert(dk);
new_flags = dk->dk_kevent.fflags;
do_resume = true;
}

TAILQ_INSERT_TAIL(&dk->dk_sources, ds, ds_list);

if (do_resume) {
dk->dk_kevent.flags |= EV_ADD;
_dispatch_kevent_resume(ds->ds_dkev, new_flags, 0);
ds->ds_is_armed = true;
}
}

The function searches for whether an identical dispatch kevent is already being observed. If it exists, new flags are merged. This also shows that a kevent is independent of a source; it does not belong exclusively to one source. If the event does not exist, the dk is inserted into the event queue. Finally, the source’s ds_list is inserted into the kevent source queue, and when flags are updated the kevent event is registered again.

    1. _dispatch_source_init_tail_queue_array
1
2
3
4
5
6
7
8
9
10
11
12
13
14
static void
_dispatch_source_init_tail_queue_array(void *context __attribute__((unused)))
{
unsigned int i;
for (i = 0; i < DSL_HASH_SIZE; i++) {
TAILQ_INIT(&_dispatch_sources[i]);
}

TAILQ_INSERT_TAIL(&_dispatch_sources[DSL_HASH(DISPATCH_TIMER_INDEX_WALL)], &_dispatch_kevent_timer[DISPATCH_TIMER_INDEX_WALL], dk_list);
TAILQ_INSERT_TAIL(&_dispatch_sources[DSL_HASH(DISPATCH_TIMER_INDEX_MACH)], &_dispatch_kevent_timer[DISPATCH_TIMER_INDEX_MACH], dk_list);

TAILQ_INSERT_TAIL(&_dispatch_sources[0], &_dispatch_kevent_data_or, dk_list);
TAILQ_INSERT_TAIL(&_dispatch_sources[0], &_dispatch_kevent_data_add, dk_list);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#define DSL_HASH_SIZE 256u  
#define DSL_HASH(x) ((x) & (DSL_HASH_SIZE - 1))
static TAILQ_HEAD(, dispatch_kevent_s) _dispatch_sources[DSL_HASH_SIZE];

#define DISPATCH_TIMER_INDEX_WALL 0
#define DISPATCH_TIMER_INDEX_MACH 1
static struct dispatch_kevent_s _dispatch_kevent_timer[] = {
{
.dk_kevent = {
.ident = DISPATCH_TIMER_INDEX_WALL,
.filter = DISPATCH_EVFILT_TIMER,
.udata = &_dispatch_kevent_timer[0],
},
.dk_sources = TAILQ_HEAD_INITIALIZER(_dispatch_kevent_timer[0].dk_sources),
},
{
.dk_kevent = {
.ident = DISPATCH_TIMER_INDEX_MACH,
.filter = DISPATCH_EVFILT_TIMER,
.udata = &_dispatch_kevent_timer[1],
},
.dk_sources = TAILQ_HEAD_INITIALIZER(_dispatch_kevent_timer[1].dk_sources),
},
};
    1. _dispatch_kevent_find
1
2
3
4
5
6
7
8
9
10
11
12
13
static dispatch_kevent_t _dispatch_kevent_find(uintptr_t ident, short filter)
{
uintptr_t hash = DSL_HASH(filter == EVFILT_MACHPORT ? MACH_PORT_INDEX(ident) : ident);
dispatch_kevent_t dki;

TAILQ_FOREACH(dki, &_dispatch_sources[hash], dk_list) {
// ident and filter must both match.
if (dki->dk_kevent.ident == ident && dki->dk_kevent.filter == filter) {
break;
}
}
return dki;
}
    1. _dispatch_kevent_resume
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
void _dispatch_kevent_resume(dispatch_kevent_t dk, uint32_t new_flags, uint32_t del_flags)
{
switch (dk->dk_kevent.filter) {
case DISPATCH_EVFILT_TIMER:
case DISPATCH_EVFILT_CUSTOM_ADD:
case DISPATCH_EVFILT_CUSTOM_OR:
// these types not registered with kevent
return;
case EVFILT_MACHPORT:
_dispatch_kevent_machport_resume(dk, new_flags, del_flags);
break;
case EVFILT_PROC:
if (dk->dk_kevent.flags & EV_ONESHOT) {
return;
}
// fall through
default:
_dispatch_update_kq(&dk->dk_kevent);
if (dk->dk_kevent.flags & EV_DISPATCH) {
dk->dk_kevent.flags &= ~EV_ADD;
}
break;
}
}

_dispatch_source_cancel_callout

This mainly performs cleanup:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
void
_dispatch_source_cancel_callout(dispatch_source_t ds)
{
ds->ds_pending_data_mask = 0;
ds->ds_pending_data = 0;
ds->ds_data = 0;

#ifdef __BLOCKS__
if (ds->ds_handler_is_block) {
Block_release(ds->ds_handler_ctxt);
ds->ds_handler_is_block = false;
ds->ds_handler_func = NULL;
ds->ds_handler_ctxt = NULL;
}
#endif

if (!ds->ds_cancel_handler) {
return;
}

if (ds->ds_cancel_is_block) {
#ifdef __BLOCKS__
dispatch_block_t b = ds->ds_cancel_handler;
if (ds->ds_atomic_flags & DSF_CANCELED) {
b();
}
Block_release(ds->ds_cancel_handler);
ds->ds_cancel_is_block = false;
#endif
}
else {
dispatch_function_t f = ds->ds_cancel_handler;
if (ds->ds_atomic_flags & DSF_CANCELED) {
f(ds->do_ctxt);
}
}

ds->ds_cancel_handler = NULL;
}

_dispatch_source_latch_and_call

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
void _dispatch_source_latch_and_call(dispatch_source_t ds)
{
unsigned long prev;

if ((ds->ds_atomic_flags & DSF_CANCELED) || (ds->do_xref_cnt == 0)) {
return;
}
prev = dispatch_atomic_xchg(&ds->ds_pending_data, 0);
if (ds->ds_is_level) {
ds->ds_data = ~prev;
} else {
ds->ds_data = prev;
}
if (dispatch_assume(prev)) {
if (ds->ds_handler_func) {
ds->ds_handler_func(ds->ds_handler_ctxt, ds);
}
}
}

The Manager Queue

We mentioned _dispatch_mgr_q in the queue article, but did not yet see where it was used. It is closely tied to sources, so this section focuses on it.

1
2
3
4
5
6
7
8
9
10
11
struct dispatch_queue_s _dispatch_mgr_q = {
.do_vtable = &_dispatch_queue_mgr_vtable,
.do_ref_cnt = DISPATCH_OBJECT_GLOBAL_REFCNT,
.do_xref_cnt = DISPATCH_OBJECT_GLOBAL_REFCNT,
.do_suspend_cnt = DISPATCH_OBJECT_SUSPEND_LOCK,
.do_targetq = &_dispatch_root_queues[DISPATCH_ROOT_QUEUE_COUNT - 1],

.dq_label = "com.apple.libdispatch-manager",
.dq_width = 1,
.dq_serialnum = 2,
};

The manager queue’s do_vtable points to _dispatch_queue_mgr_vtable:

1
2
3
4
5
6
7
static const struct dispatch_queue_vtable_s _dispatch_queue_mgr_vtable = {
.do_type = DISPATCH_QUEUE_MGR_TYPE,
.do_kind = "mgr-queue",
.do_invoke = _dispatch_mgr_invoke,
.do_debug = dispatch_queue_debug,
.do_probe = _dispatch_mgr_wakeup,
};

There is no doubt that the most important fields here are do_invoke and do_probe. They are also the entrance to understanding the lower layers of sources.

At this point, some I/O multiplexing background becomes useful:

  • Synchronous blocking, asynchronous blocking, synchronous non-blocking, and related models.
  • select, epoll, kqueue, and similar mechanisms.

_dispatch_mgr_wakeup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
static bool _dispatch_mgr_wakeup(dispatch_queue_t dq)
{
static const struct kevent kev = {
.ident = 1,
.filter = EVFILT_USER,
#ifdef EV_TRIGGER
.flags = EV_TRIGGER,
#endif
#ifdef NOTE_TRIGGER
.fflags = NOTE_TRIGGER,
#endif
};
_dispatch_update_kq(&kev);

return false;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
void
_dispatch_update_kq(const struct kevent *kev)
{
struct kevent kev_copy = *kev;
kev_copy.flags |= EV_RECEIPT;

if (kev_copy.flags & EV_DELETE) {
switch (kev_copy.filter) {
case EVFILT_READ:
if (FD_ISSET((int)kev_copy.ident, &_dispatch_rfds)) {
FD_CLR((int)kev_copy.ident, &_dispatch_rfds);
_dispatch_rfd_ptrs[kev_copy.ident] = 0;
return;
}
case EVFILT_WRITE:
if (FD_ISSET((int)kev_copy.ident, &_dispatch_wfds)) {
FD_CLR((int)kev_copy.ident, &_dispatch_wfds);
_dispatch_wfd_ptrs[kev_copy.ident] = 0;
return;
}
default:
break;
}
}

int rval = kevent(_dispatch_get_kq(), &kev_copy, 1, &kev_copy, 1, NULL);

switch (kev_copy.data) {
case 0:
return;
case EBADF:
break;
default:
switch (kev_copy.filter) {
case EVFILT_READ:
_dispatch_select_workaround = true;
FD_SET((int)kev_copy.ident, &_dispatch_rfds);
_dispatch_rfd_ptrs[kev_copy.ident] = kev_copy.udata;
break;
case EVFILT_WRITE:
_dispatch_select_workaround = true;
FD_SET((int)kev_copy.ident, &_dispatch_wfds);
_dispatch_wfd_ptrs[kev_copy.ident] = kev_copy.udata;
break;
default:
_dispatch_source_drain_kevent(&kev_copy);
break;
}
break;
}
}

If registering the event fails because kevent is reading or writing data, _dispatch_select_workaround is set to true and select is used later as a fallback. Apart from EVFILT_READ and EVFILT_WRITE, other event types call _dispatch_source_drain_kevent.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
void _dispatch_source_drain_kevent(struct kevent *ke)
{
static dispatch_once_t pred;
dispatch_kevent_t dk = ke->udata;
dispatch_source_t dsi;

if (ke->filter == EVFILT_MACHPORT) {
return _dispatch_drain_mach_messages(ke);
}

if (ke->flags & EV_ONESHOT) {
dk->dk_kevent.flags |= EV_ONESHOT;
}

TAILQ_FORACH(dsi, &dk->dk_sources, ds_list) {
_dispatch_source_merge_kevent(dsi, ke);
}
}

This iterates through every source in dk->dk_sources and calls _dispatch_source_merge_kevent, meaning the event is ready.

_dispatch_mgr_invoke

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
static dispatch_queue_t _dispatch_mgr_invoke(dispatch_queue_t dq)
{
static const struct timespec timeout_immediately = { 0, 0 };
struct timespec timeout;
const struct timespec *timeoutp;
struct timeval sel_timeout, *sel_timeoutp;
fd_set tmp_rfds, tmp_wfds;
struct kevent kev[1];
int k_cnt, k_err, i, r;

_dispatch_thread_setspecific(dispatch_queue_key, dq);

for (;;) {
_dispatch_run_timers();

timeoutp = _dispatch_get_next_timer_fire(&timeout);

if (_dispatch_select_workaround) {
FD_COPY(&_dispatch_rfds, &tmp_rfds);
FD_COPY(&_dispatch_wfds, &tmp_wfds);
if (timeoutp) {
sel_timeout.tv_sec = timeoutp->tv_sec;
sel_timeout.tv_usec = (typeof(sel_timeout.tv_usec))(timeoutp->tv_nsec / 1000u);
sel_timeoutp = &sel_timeout;
} else {
sel_timeoutp = NULL;
}

r = select(FD_SETSIZE, &tmp_rfds, &tmp_wfds, NULL, sel_timeoutp);

if (r == -1) {
if (errno != EBADF) {
dispatch_assume_zero(errno);
continue;
}
for (i = 0; i < FD_SETSIZE; i++) {
if (i == _dispatch_kq) {
continue;
}
if (!FD_ISSET(i, &_dispatch_rfds) && !FD_ISSET(i, &_dispatch_wfds)) {
continue;
}
r = dup(i);
if (r != -1) {
close(r);
} else {
FD_CLR(i, &_dispatch_rfds);
FD_CLR(i, &_dispatch_wfds);
_dispatch_rfd_ptrs[i] = 0;
_dispatch_wfd_ptrs[i] = 0;
}
}
continue;
}

if (r > 0) {
for (i = 0; i < FD_SETSIZE; i++) {
if (i == _dispatch_kq) {
continue;
}
if (FD_ISSET(i, &tmp_rfds)) {
FD_CLR(i, &_dispatch_rfds);
EV_SET(&kev[0], i, EVFILT_READ, EV_ADD|EV_ENABLE|EV_DISPATCH, 0, 1, _dispatch_rfd_ptrs[i]);
_dispatch_rfd_ptrs[i] = 0;
_dispatch_mgr_thread2(kev, 1);
}
if (FD_ISSET(i, &tmp_wfds)) {
FD_CLR(i, &_dispatch_wfds);
EV_SET(&kev[0], i, EVFILT_WRITE, EV_ADD|EV_ENABLE|EV_DISPATCH, 0, 1, _dispatch_wfd_ptrs[i]);
_dispatch_wfd_ptrs[i] = 0;
_dispatch_mgr_thread2(kev, 1);
}
}
}

timeoutp = &timeout_immediately;
}

k_cnt = kevent(_dispatch_kq, NULL, 0, kev, sizeof(kev) / sizeof(kev[0]), timeoutp);
k_err = errno;

switch (k_cnt) {
case -1:
if (k_err == EBADF) {
DISPATCH_CLIENT_CRASH("Do not close random Unix descriptors");
}
dispatch_assume_zero(k_err);
continue;
default:
_dispatch_mgr_thread2(kev, (size_t)k_cnt);
case 0:
_dispatch_force_cache_cleanup();
continue;
}
}

return NULL;
}

Both the special fallback path and the normal path eventually call _dispatch_mgr_thread2:

1
2
3
4
5
6
7
8
9
10
11
12
13
static void _dispatch_mgr_thread2(struct kevent *kev, size_t cnt)
{
size_t i;

for (i = 0; i < cnt; i++) {
if (kev[i].filter == EVFILT_USER) {
_dispatch_queue_serial_drain_till_empty(&_dispatch_mgr_q);
}
else {
_dispatch_source_drain_kevent(&kev[i]);
}
}
}
1
2
3
4
5
void _dispatch_queue_serial_drain_till_empty(dispatch_queue_t dq)
{
_dispatch_queue_drain(dq);
_dispatch_force_cache_cleanup();
}

Summary

This article is long and relatively complex, and source itself is also less familiar than other GCD APIs. The two most important ideas here are I/O multiplexing and the manager queue. Once you keep those two points in mind, the underlying principle of dispatch_source becomes much easier to understand.

-------------本文结束 感谢您的阅读-------------
If this article helped you, your support encourages me to keep writing.

Title:GCD Source Analysis 6 - dispatch_source

Author:lingyun

Published:2018-02-10 00:25

Updated:2026-09-23 00:37

Original link:https://tsuijunxi.github.io/en/2018/02/10/gcd-source-analysis-6-dispatch-source/

License: Attribution-NonCommercial-NoDerivatives 4.0 International Please keep the original link and author when sharing.