450 lines
16 KiB
Plaintext
450 lines
16 KiB
Plaintext
Read the F-ing Papers!
|
|
|
|
|
|
This document describes RCU-related publications, and is followed by
|
|
the corresponding bibtex entries. A number of the publications may
|
|
be found at http://www.rdrop.com/users/paulmck/RCU/.
|
|
|
|
The first thing resembling RCU was published in 1980, when Kung and Lehman
|
|
[Kung80] recommended use of a garbage collector to defer destruction
|
|
of nodes in a parallel binary search tree in order to simplify its
|
|
implementation. This works well in environments that have garbage
|
|
collectors, but current production garbage collectors incur significant
|
|
read-side overhead.
|
|
|
|
In 1982, Manber and Ladner [Manber82,Manber84] recommended deferring
|
|
destruction until all threads running at that time have terminated, again
|
|
for a parallel binary search tree. This approach works well in systems
|
|
with short-lived threads, such as the K42 research operating system.
|
|
However, Linux has long-lived tasks, so more is needed.
|
|
|
|
In 1986, Hennessy, Osisek, and Seigh [Hennessy89] introduced passive
|
|
serialization, which is an RCU-like mechanism that relies on the presence
|
|
of "quiescent states" in the VM/XA hypervisor that are guaranteed not
|
|
to be referencing the data structure. However, this mechanism was not
|
|
optimized for modern computer systems, which is not surprising given
|
|
that these overheads were not so expensive in the mid-80s. Nonetheless,
|
|
passive serialization appears to be the first deferred-destruction
|
|
mechanism to be used in production. Furthermore, the relevant patent has
|
|
lapsed, so this approach may be used in non-GPL software, if desired.
|
|
(In contrast, use of RCU is permitted only in software licensed under
|
|
GPL. Sorry!!!)
|
|
|
|
In 1990, Pugh [Pugh90] noted that explicitly tracking which threads
|
|
were reading a given data structure permitted deferred free to operate
|
|
in the presence of non-terminating threads. However, this explicit
|
|
tracking imposes significant read-side overhead, which is undesirable
|
|
in read-mostly situations. This algorithm does take pains to avoid
|
|
write-side contention and parallelize the other write-side overheads by
|
|
providing a fine-grained locking design, however, it would be interesting
|
|
to see how much of the performance advantage reported in 1990 remains
|
|
in 2004.
|
|
|
|
At about this same time, Adams [Adams91] described ``chaotic relaxation'',
|
|
where the normal barriers between successive iterations of convergent
|
|
numerical algorithms are relaxed, so that iteration $n$ might use
|
|
data from iteration $n-1$ or even $n-2$. This introduces error,
|
|
which typically slows convergence and thus increases the number of
|
|
iterations required. However, this increase is sometimes more than made
|
|
up for by a reduction in the number of expensive barrier operations,
|
|
which are otherwise required to synchronize the threads at the end
|
|
of each iteration. Unfortunately, chaotic relaxation requires highly
|
|
structured data, such as the matrices used in scientific programs, and
|
|
is thus inapplicable to most data structures in operating-system kernels.
|
|
|
|
In 1993, Jacobson [Jacobson93] verbally described what is perhaps the
|
|
simplest deferred-free technique: simply waiting a fixed amount of time
|
|
before freeing blocks awaiting deferred free. Jacobson did not describe
|
|
any write-side changes he might have made in this work using SGI's Irix
|
|
kernel. Aju John published a similar technique in 1995 [AjuJohn95].
|
|
This works well if there is a well-defined upper bound on the length of
|
|
time that reading threads can hold references, as there might well be in
|
|
hard real-time systems. However, if this time is exceeded, perhaps due
|
|
to preemption, excessive interrupts, or larger-than-anticipated load,
|
|
memory corruption can ensue, with no reasonable means of diagnosis.
|
|
Jacobson's technique is therefore inappropriate for use in production
|
|
operating-system kernels, except when such kernels can provide hard
|
|
real-time response guarantees for all operations.
|
|
|
|
Also in 1995, Pu et al. [Pu95a] applied a technique similar to that of Pugh's
|
|
read-side-tracking to permit replugging of algorithms within a commercial
|
|
Unix operating system. However, this replugging permitted only a single
|
|
reader at a time. The following year, this same group of researchers
|
|
extended their technique to allow for multiple readers [Cowan96a].
|
|
Their approach requires memory barriers (and thus pipeline stalls),
|
|
but reduces memory latency, contention, and locking overheads.
|
|
|
|
1995 also saw the first publication of DYNIX/ptx's RCU mechanism
|
|
[Slingwine95], which was optimized for modern CPU architectures,
|
|
and was successfully applied to a number of situations within the
|
|
DYNIX/ptx kernel. The corresponding conference paper appeared in 1998
|
|
[McKenney98].
|
|
|
|
In 1999, the Tornado and K42 groups described their "generations"
|
|
mechanism, which quite similar to RCU [Gamsa99]. These operating systems
|
|
made pervasive use of RCU in place of "existence locks", which greatly
|
|
simplifies locking hierarchies.
|
|
|
|
2001 saw the first RCU presentation involving Linux [McKenney01a]
|
|
at OLS. The resulting abundance of RCU patches was presented the
|
|
following year [McKenney02a], and use of RCU in dcache was first
|
|
described that same year [Linder02a].
|
|
|
|
Also in 2002, Michael [Michael02b,Michael02a] presented "hazard-pointer"
|
|
techniques that defer the destruction of data structures to simplify
|
|
non-blocking synchronization (wait-free synchronization, lock-free
|
|
synchronization, and obstruction-free synchronization are all examples of
|
|
non-blocking synchronization). In particular, this technique eliminates
|
|
locking, reduces contention, reduces memory latency for readers, and
|
|
parallelizes pipeline stalls and memory latency for writers. However,
|
|
these techniques still impose significant read-side overhead in the
|
|
form of memory barriers. Researchers at Sun worked along similar lines
|
|
in the same timeframe [HerlihyLM02,HerlihyLMS03]. These techniques
|
|
can be thought of as inside-out reference counts, where the count is
|
|
represented by the number of hazard pointers referencing a given data
|
|
structure (rather than the more conventional counter field within the
|
|
data structure itself).
|
|
|
|
In 2003, the K42 group described how RCU could be used to create
|
|
hot-pluggable implementations of operating-system functions. Later that
|
|
year saw a paper describing an RCU implementation of System V IPC
|
|
[Arcangeli03], and an introduction to RCU in Linux Journal [McKenney03a].
|
|
|
|
2004 has seen a Linux-Journal article on use of RCU in dcache
|
|
[McKenney04a], a performance comparison of locking to RCU on several
|
|
different CPUs [McKenney04b], a dissertation describing use of RCU in a
|
|
number of operating-system kernels [PaulEdwardMcKenneyPhD], a paper
|
|
describing how to make RCU safe for soft-realtime applications [Sarma04c],
|
|
and a paper describing SELinux performance with RCU [JamesMorris04b].
|
|
|
|
2005 has seen further adaptation of RCU to realtime use, permitting
|
|
preemption of RCU realtime critical sections [PaulMcKenney05a,
|
|
PaulMcKenney05b].
|
|
|
|
Bibtex Entries
|
|
|
|
@article{Kung80
|
|
,author="H. T. Kung and Q. Lehman"
|
|
,title="Concurrent Maintenance of Binary Search Trees"
|
|
,Year="1980"
|
|
,Month="September"
|
|
,journal="ACM Transactions on Database Systems"
|
|
,volume="5"
|
|
,number="3"
|
|
,pages="354-382"
|
|
}
|
|
|
|
@techreport{Manber82
|
|
,author="Udi Manber and Richard E. Ladner"
|
|
,title="Concurrency Control in a Dynamic Search Structure"
|
|
,institution="Department of Computer Science, University of Washington"
|
|
,address="Seattle, Washington"
|
|
,year="1982"
|
|
,number="82-01-01"
|
|
,month="January"
|
|
,pages="28"
|
|
}
|
|
|
|
@article{Manber84
|
|
,author="Udi Manber and Richard E. Ladner"
|
|
,title="Concurrency Control in a Dynamic Search Structure"
|
|
,Year="1984"
|
|
,Month="September"
|
|
,journal="ACM Transactions on Database Systems"
|
|
,volume="9"
|
|
,number="3"
|
|
,pages="439-455"
|
|
}
|
|
|
|
@techreport{Hennessy89
|
|
,author="James P. Hennessy and Damian L. Osisek and Joseph W. {Seigh II}"
|
|
,title="Passive Serialization in a Multitasking Environment"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="1989"
|
|
,number="US Patent 4,809,168 (lapsed)"
|
|
,month="February"
|
|
,pages="11"
|
|
}
|
|
|
|
@techreport{Pugh90
|
|
,author="William Pugh"
|
|
,title="Concurrent Maintenance of Skip Lists"
|
|
,institution="Institute of Advanced Computer Science Studies, Department of Computer Science, University of Maryland"
|
|
,address="College Park, Maryland"
|
|
,year="1990"
|
|
,number="CS-TR-2222.1"
|
|
,month="June"
|
|
}
|
|
|
|
@Book{Adams91
|
|
,Author="Gregory R. Adams"
|
|
,title="Concurrent Programming, Principles, and Practices"
|
|
,Publisher="Benjamin Cummins"
|
|
,Year="1991"
|
|
}
|
|
|
|
@unpublished{Jacobson93
|
|
,author="Van Jacobson"
|
|
,title="Avoid Read-Side Locking Via Delayed Free"
|
|
,year="1993"
|
|
,month="September"
|
|
,note="Verbal discussion"
|
|
}
|
|
|
|
@Conference{AjuJohn95
|
|
,Author="Aju John"
|
|
,Title="Dynamic vnodes -- Design and Implementation"
|
|
,Booktitle="{USENIX Winter 1995}"
|
|
,Publisher="USENIX Association"
|
|
,Month="January"
|
|
,Year="1995"
|
|
,pages="11-23"
|
|
,Address="New Orleans, LA"
|
|
}
|
|
|
|
@techreport{Slingwine95
|
|
,author="John D. Slingwine and Paul E. McKenney"
|
|
,title="Apparatus and Method for Achieving Reduced Overhead Mutual
|
|
Exclusion and Maintaining Coherency in a Multiprocessor System
|
|
Utilizing Execution History and Thread Monitoring"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="1995"
|
|
,number="US Patent 5,442,758 (contributed under GPL)"
|
|
,month="August"
|
|
}
|
|
|
|
@techreport{Slingwine97
|
|
,author="John D. Slingwine and Paul E. McKenney"
|
|
,title="Method for maintaining data coherency using thread
|
|
activity summaries in a multicomputer system"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="1997"
|
|
,number="US Patent 5,608,893 (contributed under GPL)"
|
|
,month="March"
|
|
}
|
|
|
|
@techreport{Slingwine98
|
|
,author="John D. Slingwine and Paul E. McKenney"
|
|
,title="Apparatus and method for achieving reduced overhead
|
|
mutual exclusion and maintaining coherency in a multiprocessor
|
|
system utilizing execution history and thread monitoring"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="1998"
|
|
,number="US Patent 5,727,209 (contributed under GPL)"
|
|
,month="March"
|
|
}
|
|
|
|
@Conference{McKenney98
|
|
,Author="Paul E. McKenney and John D. Slingwine"
|
|
,Title="Read-Copy Update: Using Execution History to Solve Concurrency
|
|
Problems"
|
|
,Booktitle="{Parallel and Distributed Computing and Systems}"
|
|
,Month="October"
|
|
,Year="1998"
|
|
,pages="509-518"
|
|
,Address="Las Vegas, NV"
|
|
}
|
|
|
|
@Conference{Gamsa99
|
|
,Author="Ben Gamsa and Orran Krieger and Jonathan Appavoo and Michael Stumm"
|
|
,Title="Tornado: Maximizing Locality and Concurrency in a Shared Memory
|
|
Multiprocessor Operating System"
|
|
,Booktitle="{Proceedings of the 3\textsuperscript{rd} Symposium on
|
|
Operating System Design and Implementation}"
|
|
,Month="February"
|
|
,Year="1999"
|
|
,pages="87-100"
|
|
,Address="New Orleans, LA"
|
|
}
|
|
|
|
@techreport{Slingwine01
|
|
,author="John D. Slingwine and Paul E. McKenney"
|
|
,title="Apparatus and method for achieving reduced overhead
|
|
mutual exclusion and maintaining coherency in a multiprocessor
|
|
system utilizing execution history and thread monitoring"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="2001"
|
|
,number="US Patent 5,219,690 (contributed under GPL)"
|
|
,month="April"
|
|
}
|
|
|
|
@Conference{McKenney01a
|
|
,Author="Paul E. McKenney and Jonathan Appavoo and Andi Kleen and
|
|
Orran Krieger and Rusty Russell and Dipankar Sarma and Maneesh Soni"
|
|
,Title="Read-Copy Update"
|
|
,Booktitle="{Ottawa Linux Symposium}"
|
|
,Month="July"
|
|
,Year="2001"
|
|
,note="Available:
|
|
\url{http://www.linuxsymposium.org/2001/abstracts/readcopy.php}
|
|
\url{http://www.rdrop.com/users/paulmck/rclock/rclock_OLS.2001.05.01c.pdf}
|
|
[Viewed June 23, 2004]"
|
|
annotation="
|
|
Described RCU, and presented some patches implementing and using it in
|
|
the Linux kernel.
|
|
"
|
|
}
|
|
|
|
@Conference{Linder02a
|
|
,Author="Hanna Linder and Dipankar Sarma and Maneesh Soni"
|
|
,Title="Scalability of the Directory Entry Cache"
|
|
,Booktitle="{Ottawa Linux Symposium}"
|
|
,Month="June"
|
|
,Year="2002"
|
|
,pages="289-300"
|
|
}
|
|
|
|
@Conference{McKenney02a
|
|
,Author="Paul E. McKenney and Dipankar Sarma and
|
|
Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell"
|
|
,Title="Read-Copy Update"
|
|
,Booktitle="{Ottawa Linux Symposium}"
|
|
,Month="June"
|
|
,Year="2002"
|
|
,pages="338-367"
|
|
,note="Available:
|
|
\url{http://www.linux.org.uk/~ajh/ols2002_proceedings.pdf.gz}
|
|
[Viewed June 23, 2004]"
|
|
}
|
|
|
|
@article{Appavoo03a
|
|
,author="J. Appavoo and K. Hui and C. A. N. Soules and R. W. Wisniewski and
|
|
D. M. {Da Silva} and O. Krieger and M. A. Auslander and D. J. Edelsohn and
|
|
B. Gamsa and G. R. Ganger and P. McKenney and M. Ostrowski and
|
|
B. Rosenburg and M. Stumm and J. Xenidis"
|
|
,title="Enabling Autonomic Behavior in Systems Software With Hot Swapping"
|
|
,Year="2003"
|
|
,Month="January"
|
|
,journal="IBM Systems Journal"
|
|
,volume="42"
|
|
,number="1"
|
|
,pages="60-76"
|
|
}
|
|
|
|
@Conference{Arcangeli03
|
|
,Author="Andrea Arcangeli and Mingming Cao and Paul E. McKenney and
|
|
Dipankar Sarma"
|
|
,Title="Using Read-Copy Update Techniques for {System V IPC} in the
|
|
{Linux} 2.5 Kernel"
|
|
,Booktitle="Proceedings of the 2003 USENIX Annual Technical Conference
|
|
(FREENIX Track)"
|
|
,Publisher="USENIX Association"
|
|
,year="2003"
|
|
,month="June"
|
|
,pages="297-310"
|
|
}
|
|
|
|
@article{McKenney03a
|
|
,author="Paul E. McKenney"
|
|
,title="Using {RCU} in the {Linux} 2.5 Kernel"
|
|
,Year="2003"
|
|
,Month="October"
|
|
,journal="Linux Journal"
|
|
,volume="1"
|
|
,number="114"
|
|
,pages="18-26"
|
|
}
|
|
|
|
@techreport{Friedberg03a
|
|
,author="Stuart A. Friedberg"
|
|
,title="Lock-Free Wild Card Search Data Structure and Method"
|
|
,institution="US Patent and Trademark Office"
|
|
,address="Washington, DC"
|
|
,year="2003"
|
|
,number="US Patent 6,662,184 (contributed under GPL)"
|
|
,month="December"
|
|
,pages="112"
|
|
}
|
|
|
|
@article{McKenney04a
|
|
,author="Paul E. McKenney and Dipankar Sarma and Maneesh Soni"
|
|
,title="Scaling dcache with {RCU}"
|
|
,Year="2004"
|
|
,Month="January"
|
|
,journal="Linux Journal"
|
|
,volume="1"
|
|
,number="118"
|
|
,pages="38-46"
|
|
}
|
|
|
|
@Conference{McKenney04b
|
|
,Author="Paul E. McKenney"
|
|
,Title="{RCU} vs. Locking Performance on Different {CPUs}"
|
|
,Booktitle="{linux.conf.au}"
|
|
,Month="January"
|
|
,Year="2004"
|
|
,Address="Adelaide, Australia"
|
|
,note="Available:
|
|
\url{http://www.linux.org.au/conf/2004/abstracts.html#90}
|
|
\url{http://www.rdrop.com/users/paulmck/rclock/lockperf.2004.01.17a.pdf}
|
|
[Viewed June 23, 2004]"
|
|
}
|
|
|
|
@phdthesis{PaulEdwardMcKenneyPhD
|
|
,author="Paul E. McKenney"
|
|
,title="Exploiting Deferred Destruction:
|
|
An Analysis of Read-Copy-Update Techniques
|
|
in Operating System Kernels"
|
|
,school="OGI School of Science and Engineering at
|
|
Oregon Health and Sciences University"
|
|
,year="2004"
|
|
,note="Available:
|
|
\url{http://www.rdrop.com/users/paulmck/RCU/RCUdissertation.2004.07.14e1.pdf}
|
|
[Viewed October 15, 2004]"
|
|
}
|
|
|
|
@Conference{Sarma04c
|
|
,Author="Dipankar Sarma and Paul E. McKenney"
|
|
,Title="Making RCU Safe for Deep Sub-Millisecond Response Realtime Applications"
|
|
,Booktitle="Proceedings of the 2004 USENIX Annual Technical Conference
|
|
(FREENIX Track)"
|
|
,Publisher="USENIX Association"
|
|
,year="2004"
|
|
,month="June"
|
|
,pages="182-191"
|
|
}
|
|
|
|
@unpublished{JamesMorris04b
|
|
,Author="James Morris"
|
|
,Title="Recent Developments in {SELinux} Kernel Performance"
|
|
,month="December"
|
|
,year="2004"
|
|
,note="Available:
|
|
\url{http://www.livejournal.com/users/james_morris/2153.html}
|
|
[Viewed December 10, 2004]"
|
|
}
|
|
|
|
@unpublished{PaulMcKenney05a
|
|
,Author="Paul E. McKenney"
|
|
,Title="{[RFC]} {RCU} and {CONFIG\_PREEMPT\_RT} progress"
|
|
,month="May"
|
|
,year="2005"
|
|
,note="Available:
|
|
\url{http://lkml.org/lkml/2005/5/9/185}
|
|
[Viewed May 13, 2005]"
|
|
,annotation="
|
|
First publication of working lock-based deferred free patches
|
|
for the CONFIG_PREEMPT_RT environment.
|
|
"
|
|
}
|
|
|
|
@conference{PaulMcKenney05b
|
|
,Author="Paul E. McKenney and Dipankar Sarma"
|
|
,Title="Towards Hard Realtime Response from the Linux Kernel on SMP Hardware"
|
|
,Booktitle="linux.conf.au 2005"
|
|
,month="April"
|
|
,year="2005"
|
|
,address="Canberra, Australia"
|
|
,note="Available:
|
|
\url{http://www.rdrop.com/users/paulmck/RCU/realtimeRCU.2005.04.23a.pdf}
|
|
[Viewed May 13, 2005]"
|
|
,annotation="
|
|
Realtime turns into making RCU yet more realtime friendly.
|
|
"
|
|
}
|