2005-04-17 06:20:36 +08:00
|
|
|
/*
|
|
|
|
|
* Macros for manipulating and testing page->flags
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifndef PAGE_FLAGS_H
|
|
|
|
|
#define PAGE_FLAGS_H
|
|
|
|
|
|
2006-06-23 17:03:06 +08:00
|
|
|
#include <linux/types.h>
|
2007-05-07 05:49:40 +08:00
|
|
|
#include <linux/mm_types.h>
|
2006-06-23 17:03:06 +08:00
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
/*
|
|
|
|
|
* Various page->flags bits:
|
|
|
|
|
*
|
|
|
|
|
* PG_reserved is set for special pages, which can never be swapped out. Some
|
|
|
|
|
* of them might not even exist (eg empty_bad_page)...
|
|
|
|
|
*
|
2006-09-26 14:31:35 +08:00
|
|
|
* The PG_private bitflag is set on pagecache pages if they contain filesystem
|
|
|
|
|
* specific data (which is normally at page->private). It can be used by
|
|
|
|
|
* private allocations for its own usage.
|
2005-04-17 06:20:36 +08:00
|
|
|
*
|
2006-09-26 14:31:35 +08:00
|
|
|
* During initiation of disk I/O, PG_locked is set. This bit is set before I/O
|
|
|
|
|
* and cleared when writeback _starts_ or when read _completes_. PG_writeback
|
|
|
|
|
* is set before writeback starts and cleared when it finishes.
|
|
|
|
|
*
|
|
|
|
|
* PG_locked also pins a page in pagecache, and blocks truncation of the file
|
|
|
|
|
* while it is held.
|
|
|
|
|
*
|
|
|
|
|
* page_waitqueue(page) is a wait queue of all tasks waiting for the page
|
|
|
|
|
* to become unlocked.
|
2005-04-17 06:20:36 +08:00
|
|
|
*
|
|
|
|
|
* PG_uptodate tells whether the page's contents is valid. When a read
|
|
|
|
|
* completes, the page becomes uptodate, unless a disk I/O error happened.
|
|
|
|
|
*
|
2006-09-26 14:31:35 +08:00
|
|
|
* PG_referenced, PG_reclaim are used for page reclaim for anonymous and
|
|
|
|
|
* file-backed pagecache (see mm/vmscan.c).
|
2005-04-17 06:20:36 +08:00
|
|
|
*
|
|
|
|
|
* PG_error is set to indicate that an I/O error occurred on this page.
|
|
|
|
|
*
|
|
|
|
|
* PG_arch_1 is an architecture specific page state bit. The generic code
|
|
|
|
|
* guarantees that this bit is cleared for a page when it first is entered into
|
|
|
|
|
* the page cache.
|
|
|
|
|
*
|
|
|
|
|
* PG_highmem pages are not permanently mapped into the kernel virtual address
|
|
|
|
|
* space, they need to be kmapped separately for doing IO on the pages. The
|
|
|
|
|
* struct page (these bits with information) are always mapped into kernel
|
|
|
|
|
* address space...
|
2006-09-26 14:31:35 +08:00
|
|
|
*
|
|
|
|
|
* PG_buddy is set to indicate that the page is free and in the buddy system
|
|
|
|
|
* (see mm/page_alloc.c).
|
|
|
|
|
*
|
2005-04-17 06:20:36 +08:00
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Don't use the *_dontuse flags. Use the macros. Otherwise you'll break
|
2006-04-11 13:53:01 +08:00
|
|
|
* locked- and dirty-page accounting.
|
|
|
|
|
*
|
|
|
|
|
* The page flags field is split into two parts, the main flags area
|
|
|
|
|
* which extends from the low bits upwards, and the fields area which
|
|
|
|
|
* extends from the high bits downwards.
|
|
|
|
|
*
|
|
|
|
|
* | FIELD | ... | FLAGS |
|
|
|
|
|
* N-1 ^ 0
|
|
|
|
|
* (N-FLAGS_RESERVED)
|
|
|
|
|
*
|
|
|
|
|
* The fields area is reserved for fields mapping zone, node and SPARSEMEM
|
|
|
|
|
* section. The boundry between these two areas is defined by
|
|
|
|
|
* FLAGS_RESERVED which defines the width of the fields section
|
|
|
|
|
* (see linux/mmzone.h). New flags must _not_ overlap with this area.
|
2005-04-17 06:20:36 +08:00
|
|
|
*/
|
|
|
|
|
#define PG_locked 0 /* Page is locked. Don't touch. */
|
|
|
|
|
#define PG_error 1
|
|
|
|
|
#define PG_referenced 2
|
|
|
|
|
#define PG_uptodate 3
|
|
|
|
|
|
|
|
|
|
#define PG_dirty 4
|
|
|
|
|
#define PG_lru 5
|
|
|
|
|
#define PG_active 6
|
|
|
|
|
#define PG_slab 7 /* slab debug (Suparna wants this) */
|
|
|
|
|
|
2007-03-01 12:12:27 +08:00
|
|
|
#define PG_owner_priv_1 8 /* Owner use. If pagecache, fs may use*/
|
2005-06-22 08:14:52 +08:00
|
|
|
#define PG_arch_1 9
|
|
|
|
|
#define PG_reserved 10
|
2006-09-26 14:31:35 +08:00
|
|
|
#define PG_private 11 /* If pagecache, has fs-private data */
|
2005-06-22 08:14:52 +08:00
|
|
|
|
|
|
|
|
#define PG_writeback 12 /* Page is under writeback */
|
|
|
|
|
#define PG_compound 14 /* Part of a compound page */
|
|
|
|
|
#define PG_swapcache 15 /* Swap page: swp_entry_t in private */
|
|
|
|
|
|
|
|
|
|
#define PG_mappedtodisk 16 /* Has blocks allocated on-disk */
|
|
|
|
|
#define PG_reclaim 17 /* To be reclaimed asap */
|
2006-04-10 09:21:48 +08:00
|
|
|
#define PG_buddy 19 /* Page is free, on buddy lists */
|
|
|
|
|
|
mm: share PG_readahead and PG_reclaim
Share the same page flag bit for PG_readahead and PG_reclaim.
One is used only on file reads, another is only for emergency writes. One
is used mostly for fresh/young pages, another is for old pages.
Combinations of possible interactions are:
a) clear PG_reclaim => implicit clear of PG_readahead
it will delay an asynchronous readahead into a synchronous one
it actually does _good_ for readahead:
the pages will be reclaimed soon, it's readahead thrashing!
in this case, synchronous readahead makes more sense.
b) clear PG_readahead => implicit clear of PG_reclaim
one(and only one) page will not be reclaimed in time
it can be avoided by checking PageWriteback(page) in readahead first
c) set PG_reclaim => implicit set of PG_readahead
will confuse readahead and make it restart the size rampup process
it's a trivial problem, and can mostly be avoided by checking
PageWriteback(page) first in readahead
d) set PG_readahead => implicit set of PG_reclaim
PG_readahead will never be set on already cached pages.
PG_reclaim will always be cleared on dirtying a page.
so not a problem.
In summary,
a) we get better behavior
b,d) possible interactions can be avoided
c) racy condition exists that might affect readahead, but the chance
is _really_ low, and the hurt on readahead is trivial.
Compound pages also use PG_reclaim, but for now they do not interact with
reclaim/readahead code.
Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-07-19 16:48:07 +08:00
|
|
|
/* PG_readahead is only used for file reads; PG_reclaim is only for writes */
|
|
|
|
|
#define PG_readahead PG_reclaim /* Reminder to do async read-ahead */
|
|
|
|
|
|
2007-03-01 12:12:27 +08:00
|
|
|
/* PG_owner_priv_1 users should have descriptive aliases */
|
|
|
|
|
#define PG_checked PG_owner_priv_1 /* Used by some filesystems */
|
2007-07-18 09:37:05 +08:00
|
|
|
#define PG_pinned PG_owner_priv_1 /* Xen pinned pagetable */
|
2006-06-23 17:03:06 +08:00
|
|
|
|
|
|
|
|
#if (BITS_PER_LONG > 32)
|
|
|
|
|
/*
|
|
|
|
|
* 64-bit-only flags build down from bit 31
|
|
|
|
|
*
|
|
|
|
|
* 32 bit -------------------------------| FIELDS | FLAGS |
|
|
|
|
|
* 64 bit | FIELDS | ?????? FLAGS |
|
|
|
|
|
* 63 32 0
|
|
|
|
|
*/
|
|
|
|
|
#define PG_uncached 31 /* Page has been mapped as uncached */
|
|
|
|
|
#endif
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Manipulation of page state flags
|
|
|
|
|
*/
|
|
|
|
|
#define PageLocked(page) \
|
|
|
|
|
test_bit(PG_locked, &(page)->flags)
|
|
|
|
|
#define SetPageLocked(page) \
|
|
|
|
|
set_bit(PG_locked, &(page)->flags)
|
|
|
|
|
#define TestSetPageLocked(page) \
|
|
|
|
|
test_and_set_bit(PG_locked, &(page)->flags)
|
|
|
|
|
#define ClearPageLocked(page) \
|
|
|
|
|
clear_bit(PG_locked, &(page)->flags)
|
|
|
|
|
#define TestClearPageLocked(page) \
|
|
|
|
|
test_and_clear_bit(PG_locked, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define PageError(page) test_bit(PG_error, &(page)->flags)
|
|
|
|
|
#define SetPageError(page) set_bit(PG_error, &(page)->flags)
|
|
|
|
|
#define ClearPageError(page) clear_bit(PG_error, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define PageReferenced(page) test_bit(PG_referenced, &(page)->flags)
|
|
|
|
|
#define SetPageReferenced(page) set_bit(PG_referenced, &(page)->flags)
|
|
|
|
|
#define ClearPageReferenced(page) clear_bit(PG_referenced, &(page)->flags)
|
|
|
|
|
#define TestClearPageReferenced(page) test_and_clear_bit(PG_referenced, &(page)->flags)
|
|
|
|
|
|
mm: fix PageUptodate data race
After running SetPageUptodate, preceeding stores to the page contents to
actually bring it uptodate may not be ordered with the store to set the
page uptodate.
Therefore, another CPU which checks PageUptodate is true, then reads the
page contents can get stale data.
Fix this by having an smp_wmb before SetPageUptodate, and smp_rmb after
PageUptodate.
Many places that test PageUptodate, do so with the page locked, and this
would be enough to ensure memory ordering in those places if
SetPageUptodate were only called while the page is locked. Unfortunately
that is not always the case for some filesystems, but it could be an idea
for the future.
Also bring the handling of anonymous page uptodateness in line with that of
file backed page management, by marking anon pages as uptodate when they
_are_ uptodate, rather than when our implementation requires that they be
marked as such. Doing allows us to get rid of the smp_wmb's in the page
copying functions, which were especially added for anonymous pages for an
analogous memory ordering problem. Both file and anonymous pages are
handled with the same barriers.
FAQ:
Q. Why not do this in flush_dcache_page?
A. Firstly, flush_dcache_page handles only one side (the smb side) of the
ordering protocol; we'd still need smp_rmb somewhere. Secondly, hiding away
memory barriers in a completely unrelated function is nasty; at least in the
PageUptodate macros, they are located together with (half) the operations
involved in the ordering. Thirdly, the smp_wmb is only required when first
bringing the page uptodate, wheras flush_dcache_page should be called each time
it is written to through the kernel mapping. It is logically the wrong place to
put it.
Q. Why does this increase my text size / reduce my performance / etc.
A. Because it is adding the necessary instructions to eliminate the data-race.
Q. Can it be improved?
A. Yes, eg. if you were to create a rule that all SetPageUptodate operations
run under the page lock, we could avoid the smp_rmb places where PageUptodate
is queried under the page lock. Requires audit of all filesystems and at least
some would need reworking. That's great you're interested, I'm eagerly awaiting
your patches.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:34 +08:00
|
|
|
static inline int PageUptodate(struct page *page)
|
|
|
|
|
{
|
|
|
|
|
int ret = test_bit(PG_uptodate, &(page)->flags);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Must ensure that the data we read out of the page is loaded
|
|
|
|
|
* _after_ we've loaded page->flags to check for PageUptodate.
|
|
|
|
|
* We can skip the barrier if the page is not uptodate, because
|
|
|
|
|
* we wouldn't be reading anything from it.
|
|
|
|
|
*
|
|
|
|
|
* See SetPageUptodate() for the other side of the story.
|
|
|
|
|
*/
|
|
|
|
|
if (ret)
|
|
|
|
|
smp_rmb();
|
|
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void __SetPageUptodate(struct page *page)
|
|
|
|
|
{
|
|
|
|
|
smp_wmb();
|
|
|
|
|
__set_bit(PG_uptodate, &(page)->flags);
|
2006-06-30 16:55:32 +08:00
|
|
|
#ifdef CONFIG_S390
|
mm: fix PageUptodate data race
After running SetPageUptodate, preceeding stores to the page contents to
actually bring it uptodate may not be ordered with the store to set the
page uptodate.
Therefore, another CPU which checks PageUptodate is true, then reads the
page contents can get stale data.
Fix this by having an smp_wmb before SetPageUptodate, and smp_rmb after
PageUptodate.
Many places that test PageUptodate, do so with the page locked, and this
would be enough to ensure memory ordering in those places if
SetPageUptodate were only called while the page is locked. Unfortunately
that is not always the case for some filesystems, but it could be an idea
for the future.
Also bring the handling of anonymous page uptodateness in line with that of
file backed page management, by marking anon pages as uptodate when they
_are_ uptodate, rather than when our implementation requires that they be
marked as such. Doing allows us to get rid of the smp_wmb's in the page
copying functions, which were especially added for anonymous pages for an
analogous memory ordering problem. Both file and anonymous pages are
handled with the same barriers.
FAQ:
Q. Why not do this in flush_dcache_page?
A. Firstly, flush_dcache_page handles only one side (the smb side) of the
ordering protocol; we'd still need smp_rmb somewhere. Secondly, hiding away
memory barriers in a completely unrelated function is nasty; at least in the
PageUptodate macros, they are located together with (half) the operations
involved in the ordering. Thirdly, the smp_wmb is only required when first
bringing the page uptodate, wheras flush_dcache_page should be called each time
it is written to through the kernel mapping. It is logically the wrong place to
put it.
Q. Why does this increase my text size / reduce my performance / etc.
A. Because it is adding the necessary instructions to eliminate the data-race.
Q. Can it be improved?
A. Yes, eg. if you were to create a rule that all SetPageUptodate operations
run under the page lock, we could avoid the smp_rmb places where PageUptodate
is queried under the page lock. Requires audit of all filesystems and at least
some would need reworking. That's great you're interested, I'm eagerly awaiting
your patches.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:34 +08:00
|
|
|
page_clear_dirty(page);
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
2006-09-29 16:58:41 +08:00
|
|
|
static inline void SetPageUptodate(struct page *page)
|
|
|
|
|
{
|
mm: fix PageUptodate data race
After running SetPageUptodate, preceeding stores to the page contents to
actually bring it uptodate may not be ordered with the store to set the
page uptodate.
Therefore, another CPU which checks PageUptodate is true, then reads the
page contents can get stale data.
Fix this by having an smp_wmb before SetPageUptodate, and smp_rmb after
PageUptodate.
Many places that test PageUptodate, do so with the page locked, and this
would be enough to ensure memory ordering in those places if
SetPageUptodate were only called while the page is locked. Unfortunately
that is not always the case for some filesystems, but it could be an idea
for the future.
Also bring the handling of anonymous page uptodateness in line with that of
file backed page management, by marking anon pages as uptodate when they
_are_ uptodate, rather than when our implementation requires that they be
marked as such. Doing allows us to get rid of the smp_wmb's in the page
copying functions, which were especially added for anonymous pages for an
analogous memory ordering problem. Both file and anonymous pages are
handled with the same barriers.
FAQ:
Q. Why not do this in flush_dcache_page?
A. Firstly, flush_dcache_page handles only one side (the smb side) of the
ordering protocol; we'd still need smp_rmb somewhere. Secondly, hiding away
memory barriers in a completely unrelated function is nasty; at least in the
PageUptodate macros, they are located together with (half) the operations
involved in the ordering. Thirdly, the smp_wmb is only required when first
bringing the page uptodate, wheras flush_dcache_page should be called each time
it is written to through the kernel mapping. It is logically the wrong place to
put it.
Q. Why does this increase my text size / reduce my performance / etc.
A. Because it is adding the necessary instructions to eliminate the data-race.
Q. Can it be improved?
A. Yes, eg. if you were to create a rule that all SetPageUptodate operations
run under the page lock, we could avoid the smp_rmb places where PageUptodate
is queried under the page lock. Requires audit of all filesystems and at least
some would need reworking. That's great you're interested, I'm eagerly awaiting
your patches.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:34 +08:00
|
|
|
#ifdef CONFIG_S390
|
2006-09-29 16:58:41 +08:00
|
|
|
if (!test_and_set_bit(PG_uptodate, &page->flags))
|
2007-04-27 22:01:57 +08:00
|
|
|
page_clear_dirty(page);
|
2006-06-30 16:55:32 +08:00
|
|
|
#else
|
mm: fix PageUptodate data race
After running SetPageUptodate, preceeding stores to the page contents to
actually bring it uptodate may not be ordered with the store to set the
page uptodate.
Therefore, another CPU which checks PageUptodate is true, then reads the
page contents can get stale data.
Fix this by having an smp_wmb before SetPageUptodate, and smp_rmb after
PageUptodate.
Many places that test PageUptodate, do so with the page locked, and this
would be enough to ensure memory ordering in those places if
SetPageUptodate were only called while the page is locked. Unfortunately
that is not always the case for some filesystems, but it could be an idea
for the future.
Also bring the handling of anonymous page uptodateness in line with that of
file backed page management, by marking anon pages as uptodate when they
_are_ uptodate, rather than when our implementation requires that they be
marked as such. Doing allows us to get rid of the smp_wmb's in the page
copying functions, which were especially added for anonymous pages for an
analogous memory ordering problem. Both file and anonymous pages are
handled with the same barriers.
FAQ:
Q. Why not do this in flush_dcache_page?
A. Firstly, flush_dcache_page handles only one side (the smb side) of the
ordering protocol; we'd still need smp_rmb somewhere. Secondly, hiding away
memory barriers in a completely unrelated function is nasty; at least in the
PageUptodate macros, they are located together with (half) the operations
involved in the ordering. Thirdly, the smp_wmb is only required when first
bringing the page uptodate, wheras flush_dcache_page should be called each time
it is written to through the kernel mapping. It is logically the wrong place to
put it.
Q. Why does this increase my text size / reduce my performance / etc.
A. Because it is adding the necessary instructions to eliminate the data-race.
Q. Can it be improved?
A. Yes, eg. if you were to create a rule that all SetPageUptodate operations
run under the page lock, we could avoid the smp_rmb places where PageUptodate
is queried under the page lock. Requires audit of all filesystems and at least
some would need reworking. That's great you're interested, I'm eagerly awaiting
your patches.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:34 +08:00
|
|
|
/*
|
|
|
|
|
* Memory barrier must be issued before setting the PG_uptodate bit,
|
|
|
|
|
* so that all previous stores issued in order to bring the page
|
|
|
|
|
* uptodate are actually visible before PageUptodate becomes true.
|
|
|
|
|
*
|
|
|
|
|
* s390 doesn't need an explicit smp_wmb here because the test and
|
|
|
|
|
* set bit already provides full barriers.
|
|
|
|
|
*/
|
|
|
|
|
smp_wmb();
|
|
|
|
|
set_bit(PG_uptodate, &(page)->flags);
|
2005-04-17 06:20:36 +08:00
|
|
|
#endif
|
mm: fix PageUptodate data race
After running SetPageUptodate, preceeding stores to the page contents to
actually bring it uptodate may not be ordered with the store to set the
page uptodate.
Therefore, another CPU which checks PageUptodate is true, then reads the
page contents can get stale data.
Fix this by having an smp_wmb before SetPageUptodate, and smp_rmb after
PageUptodate.
Many places that test PageUptodate, do so with the page locked, and this
would be enough to ensure memory ordering in those places if
SetPageUptodate were only called while the page is locked. Unfortunately
that is not always the case for some filesystems, but it could be an idea
for the future.
Also bring the handling of anonymous page uptodateness in line with that of
file backed page management, by marking anon pages as uptodate when they
_are_ uptodate, rather than when our implementation requires that they be
marked as such. Doing allows us to get rid of the smp_wmb's in the page
copying functions, which were especially added for anonymous pages for an
analogous memory ordering problem. Both file and anonymous pages are
handled with the same barriers.
FAQ:
Q. Why not do this in flush_dcache_page?
A. Firstly, flush_dcache_page handles only one side (the smb side) of the
ordering protocol; we'd still need smp_rmb somewhere. Secondly, hiding away
memory barriers in a completely unrelated function is nasty; at least in the
PageUptodate macros, they are located together with (half) the operations
involved in the ordering. Thirdly, the smp_wmb is only required when first
bringing the page uptodate, wheras flush_dcache_page should be called each time
it is written to through the kernel mapping. It is logically the wrong place to
put it.
Q. Why does this increase my text size / reduce my performance / etc.
A. Because it is adding the necessary instructions to eliminate the data-race.
Q. Can it be improved?
A. Yes, eg. if you were to create a rule that all SetPageUptodate operations
run under the page lock, we could avoid the smp_rmb places where PageUptodate
is queried under the page lock. Requires audit of all filesystems and at least
some would need reworking. That's great you're interested, I'm eagerly awaiting
your patches.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-05 14:29:34 +08:00
|
|
|
}
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
#define ClearPageUptodate(page) clear_bit(PG_uptodate, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define PageDirty(page) test_bit(PG_dirty, &(page)->flags)
|
|
|
|
|
#define SetPageDirty(page) set_bit(PG_dirty, &(page)->flags)
|
|
|
|
|
#define TestSetPageDirty(page) test_and_set_bit(PG_dirty, &(page)->flags)
|
|
|
|
|
#define ClearPageDirty(page) clear_bit(PG_dirty, &(page)->flags)
|
2005-09-04 06:54:50 +08:00
|
|
|
#define __ClearPageDirty(page) __clear_bit(PG_dirty, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
#define TestClearPageDirty(page) test_and_clear_bit(PG_dirty, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define PageLRU(page) test_bit(PG_lru, &(page)->flags)
|
2006-03-22 16:07:59 +08:00
|
|
|
#define SetPageLRU(page) set_bit(PG_lru, &(page)->flags)
|
|
|
|
|
#define ClearPageLRU(page) clear_bit(PG_lru, &(page)->flags)
|
2006-03-22 16:08:00 +08:00
|
|
|
#define __ClearPageLRU(page) __clear_bit(PG_lru, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
#define PageActive(page) test_bit(PG_active, &(page)->flags)
|
|
|
|
|
#define SetPageActive(page) set_bit(PG_active, &(page)->flags)
|
|
|
|
|
#define ClearPageActive(page) clear_bit(PG_active, &(page)->flags)
|
2006-03-22 16:08:00 +08:00
|
|
|
#define __ClearPageActive(page) __clear_bit(PG_active, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
#define PageSlab(page) test_bit(PG_slab, &(page)->flags)
|
2006-03-22 16:08:02 +08:00
|
|
|
#define __SetPageSlab(page) __set_bit(PG_slab, &(page)->flags)
|
|
|
|
|
#define __ClearPageSlab(page) __clear_bit(PG_slab, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_HIGHMEM
|
2005-06-22 08:14:52 +08:00
|
|
|
#define PageHighMem(page) is_highmem(page_zone(page))
|
2005-04-17 06:20:36 +08:00
|
|
|
#else
|
|
|
|
|
#define PageHighMem(page) 0 /* needed to optimize away at compile time */
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#define PageChecked(page) test_bit(PG_checked, &(page)->flags)
|
|
|
|
|
#define SetPageChecked(page) set_bit(PG_checked, &(page)->flags)
|
|
|
|
|
#define ClearPageChecked(page) clear_bit(PG_checked, &(page)->flags)
|
|
|
|
|
|
2007-07-18 09:37:05 +08:00
|
|
|
#define PagePinned(page) test_bit(PG_pinned, &(page)->flags)
|
|
|
|
|
#define SetPagePinned(page) set_bit(PG_pinned, &(page)->flags)
|
|
|
|
|
#define ClearPagePinned(page) clear_bit(PG_pinned, &(page)->flags)
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
#define PageReserved(page) test_bit(PG_reserved, &(page)->flags)
|
|
|
|
|
#define SetPageReserved(page) set_bit(PG_reserved, &(page)->flags)
|
|
|
|
|
#define ClearPageReserved(page) clear_bit(PG_reserved, &(page)->flags)
|
|
|
|
|
#define __ClearPageReserved(page) __clear_bit(PG_reserved, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define SetPagePrivate(page) set_bit(PG_private, &(page)->flags)
|
|
|
|
|
#define ClearPagePrivate(page) clear_bit(PG_private, &(page)->flags)
|
|
|
|
|
#define PagePrivate(page) test_bit(PG_private, &(page)->flags)
|
|
|
|
|
#define __SetPagePrivate(page) __set_bit(PG_private, &(page)->flags)
|
|
|
|
|
#define __ClearPagePrivate(page) __clear_bit(PG_private, &(page)->flags)
|
|
|
|
|
|
2007-07-19 16:49:17 +08:00
|
|
|
/*
|
|
|
|
|
* Only test-and-set exist for PG_writeback. The unconditional operators are
|
|
|
|
|
* risky: they bypass page accounting.
|
|
|
|
|
*/
|
2005-04-17 06:20:36 +08:00
|
|
|
#define PageWriteback(page) test_bit(PG_writeback, &(page)->flags)
|
2007-07-19 16:49:17 +08:00
|
|
|
#define TestSetPageWriteback(page) test_and_set_bit(PG_writeback, \
|
|
|
|
|
&(page)->flags)
|
|
|
|
|
#define TestClearPageWriteback(page) test_and_clear_bit(PG_writeback, \
|
|
|
|
|
&(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2006-04-10 09:21:48 +08:00
|
|
|
#define PageBuddy(page) test_bit(PG_buddy, &(page)->flags)
|
|
|
|
|
#define __SetPageBuddy(page) __set_bit(PG_buddy, &(page)->flags)
|
|
|
|
|
#define __ClearPageBuddy(page) __clear_bit(PG_buddy, &(page)->flags)
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
#define PageMappedToDisk(page) test_bit(PG_mappedtodisk, &(page)->flags)
|
|
|
|
|
#define SetPageMappedToDisk(page) set_bit(PG_mappedtodisk, &(page)->flags)
|
|
|
|
|
#define ClearPageMappedToDisk(page) clear_bit(PG_mappedtodisk, &(page)->flags)
|
2007-07-19 16:47:55 +08:00
|
|
|
|
|
|
|
|
#define PageReadahead(page) test_bit(PG_readahead, &(page)->flags)
|
|
|
|
|
#define SetPageReadahead(page) set_bit(PG_readahead, &(page)->flags)
|
|
|
|
|
#define ClearPageReadahead(page) clear_bit(PG_readahead, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
#define PageReclaim(page) test_bit(PG_reclaim, &(page)->flags)
|
|
|
|
|
#define SetPageReclaim(page) set_bit(PG_reclaim, &(page)->flags)
|
|
|
|
|
#define ClearPageReclaim(page) clear_bit(PG_reclaim, &(page)->flags)
|
|
|
|
|
#define TestClearPageReclaim(page) test_and_clear_bit(PG_reclaim, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
#define PageCompound(page) test_bit(PG_compound, &(page)->flags)
|
2006-03-22 16:08:01 +08:00
|
|
|
#define __SetPageCompound(page) __set_bit(PG_compound, &(page)->flags)
|
|
|
|
|
#define __ClearPageCompound(page) __clear_bit(PG_compound, &(page)->flags)
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2007-05-07 05:49:39 +08:00
|
|
|
/*
|
2007-05-07 05:49:40 +08:00
|
|
|
* PG_reclaim is used in combination with PG_compound to mark the
|
|
|
|
|
* head and tail of a compound page
|
|
|
|
|
*
|
|
|
|
|
* PG_compound & PG_reclaim => Tail page
|
|
|
|
|
* PG_compound & ~PG_reclaim => Head page
|
2007-05-07 05:49:39 +08:00
|
|
|
*/
|
2007-05-07 05:49:40 +08:00
|
|
|
|
|
|
|
|
#define PG_head_tail_mask ((1L << PG_compound) | (1L << PG_reclaim))
|
|
|
|
|
|
2008-02-22 00:12:45 +08:00
|
|
|
#define PageTail(page) (((page)->flags & PG_head_tail_mask) \
|
2007-05-07 05:49:40 +08:00
|
|
|
== PG_head_tail_mask)
|
|
|
|
|
|
|
|
|
|
static inline void __SetPageTail(struct page *page)
|
|
|
|
|
{
|
|
|
|
|
page->flags |= PG_head_tail_mask;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void __ClearPageTail(struct page *page)
|
|
|
|
|
{
|
|
|
|
|
page->flags &= ~PG_head_tail_mask;
|
|
|
|
|
}
|
|
|
|
|
|
2008-02-22 00:12:45 +08:00
|
|
|
#define PageHead(page) (((page)->flags & PG_head_tail_mask) \
|
2007-05-07 05:49:40 +08:00
|
|
|
== (1L << PG_compound))
|
|
|
|
|
#define __SetPageHead(page) __SetPageCompound(page)
|
|
|
|
|
#define __ClearPageHead(page) __ClearPageCompound(page)
|
2007-05-07 05:49:39 +08:00
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
#ifdef CONFIG_SWAP
|
|
|
|
|
#define PageSwapCache(page) test_bit(PG_swapcache, &(page)->flags)
|
|
|
|
|
#define SetPageSwapCache(page) set_bit(PG_swapcache, &(page)->flags)
|
|
|
|
|
#define ClearPageSwapCache(page) clear_bit(PG_swapcache, &(page)->flags)
|
|
|
|
|
#else
|
|
|
|
|
#define PageSwapCache(page) 0
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#define PageUncached(page) test_bit(PG_uncached, &(page)->flags)
|
|
|
|
|
#define SetPageUncached(page) set_bit(PG_uncached, &(page)->flags)
|
|
|
|
|
#define ClearPageUncached(page) clear_bit(PG_uncached, &(page)->flags)
|
|
|
|
|
|
|
|
|
|
struct page; /* forward declaration */
|
|
|
|
|
|
VM: Remove "clear_page_dirty()" and "test_clear_page_dirty()" functions
They were horribly easy to mis-use because of their tempting naming, and
they also did way more than any users of them generally wanted them to
do.
A dirty page can become clean under two circumstances:
(a) when we write it out. We have "clear_page_dirty_for_io()" for
this, and that function remains unchanged.
In the "for IO" case it is not sufficient to just clear the dirty
bit, you also have to mark the page as being under writeback etc.
(b) when we actually remove a page due to it becoming inaccessible to
users, notably because it was truncate()'d away or the file (or
metadata) no longer exists, and we thus want to cancel any
outstanding dirty state.
For the (b) case, we now introduce "cancel_dirty_page()", which only
touches the page state itself, and verifies that the page is not mapped
(since cancelling writes on a mapped page would be actively wrong as it
is still accessible to users).
Some filesystems need to be fixed up for this: CIFS, FUSE, JFS,
ReiserFS, XFS all use the old confusing functions, and will be fixed
separately in subsequent commits (with some of them just removing the
offending logic, and others using clear_page_dirty_for_io()).
This was confirmed by Martin Michlmayr to fix the apt database
corruption on ARM.
Cc: Martin Michlmayr <tbm@cyrius.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Arjan van de Ven <arjan@infradead.org>
Cc: Andrei Popa <andrei.popa@i-neo.ro>
Cc: Andrew Morton <akpm@osdl.org>
Cc: Dave Kleikamp <shaggy@linux.vnet.ibm.com>
Cc: Gordon Farquharson <gordonfarquharson@gmail.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Trond Myklebust <trond.myklebust@fys.uio.no>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-21 05:46:42 +08:00
|
|
|
extern void cancel_dirty_page(struct page *page, unsigned int account_size);
|
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
int test_clear_page_writeback(struct page *page);
|
|
|
|
|
int test_set_page_writeback(struct page *page);
|
|
|
|
|
|
|
|
|
|
static inline void set_page_writeback(struct page *page)
|
|
|
|
|
{
|
|
|
|
|
test_set_page_writeback(page);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#endif /* PAGE_FLAGS_H */
|
