Lesson 35 - Get Compute Auth Token Working

This commit is contained in:
Norman Lansing
2026-02-28 12:32:28 -05:00
parent 1d477ee42a
commit 4fde462bce
7743 changed files with 1397833 additions and 18 deletions

View File

@@ -0,0 +1,353 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
---------------- END OF Gnu General Public License ----------------
The source code of Threading Building Blocks is distributed under version 2
of the GNU General Public License, with the so-called "runtime exception,"
as follows (or see any header or implementation file):
As a special exception, you may use this file as part of a free software
library without restriction. Specifically, if other files instantiate
templates or use macros or inline functions from this file, or you compile
this file and link it with other files to produce an executable, this
file does not by itself cause the resulting executable to be covered by
the GNU General Public License. This exception does not however
invalidate any other reasons why the executable file might be covered by
the GNU General Public License.

View File

@@ -0,0 +1,3 @@
This is a partial copy of the Intel TBB open source version.
The version taken is 4.3, obtained from https://www.threadingbuildingblocks.org/download
The files in this directory consist of the files taken from src/tbb and include/tbb

View File

@@ -0,0 +1,202 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__aggregator_H
#define __TBB__aggregator_H
#if !TBB_PREVIEW_AGGREGATOR
#error Set TBB_PREVIEW_AGGREGATOR before including aggregator.h
#endif
#include "atomic.h"
#include "tbb_profiling.h"
namespace tbb {
namespace interface6 {
using namespace tbb::internal;
class aggregator_operation {
template<typename handler_type> friend class aggregator_ext;
uintptr_t status;
aggregator_operation* my_next;
public:
enum aggregator_operation_status { agg_waiting=0, agg_finished };
aggregator_operation() : status(agg_waiting), my_next(NULL) {}
/// Call start before handling this operation
void start() { call_itt_notify(acquired, &status); }
/// Call finish when done handling this operation
/** The operation will be released to its originating thread, and possibly deleted. */
void finish() { itt_store_word_with_release(status, uintptr_t(agg_finished)); }
aggregator_operation* next() { return itt_hide_load_word(my_next);}
void set_next(aggregator_operation* n) { itt_hide_store_word(my_next, n); }
};
namespace internal {
class basic_operation_base : public aggregator_operation {
friend class basic_handler;
virtual void apply_body() = 0;
public:
basic_operation_base() : aggregator_operation() {}
virtual ~basic_operation_base() {}
};
template<typename Body>
class basic_operation : public basic_operation_base, no_assign {
const Body& my_body;
/*override*/ void apply_body() { my_body(); }
public:
basic_operation(const Body& b) : basic_operation_base(), my_body(b) {}
};
class basic_handler {
public:
basic_handler() {}
void operator()(aggregator_operation* op_list) const {
while (op_list) {
// ITT note: &(op_list->status) tag is used to cover accesses to the operation data.
// The executing thread "acquires" the tag (see start()) and then performs
// the associated operation w/o triggering a race condition diagnostics.
// A thread that created the operation is waiting for its status (see execute_impl()),
// so when this thread is done with the operation, it will "release" the tag
// and update the status (see finish()) to give control back to the waiting thread.
basic_operation_base& request = static_cast<basic_operation_base&>(*op_list);
// IMPORTANT: need to advance op_list to op_list->next() before calling request.finish()
op_list = op_list->next();
request.start();
request.apply_body();
request.finish();
}
}
};
} // namespace internal
//! Aggregator base class and expert interface
/** An aggregator for collecting operations coming from multiple sources and executing
them serially on a single thread. */
template <typename handler_type>
class aggregator_ext : tbb::internal::no_copy {
public:
aggregator_ext(const handler_type& h) : handler_busy(0), handle_operations(h) { mailbox = NULL; }
//! EXPERT INTERFACE: Enter a user-made operation into the aggregator's mailbox.
/** Details of user-made operations must be handled by user-provided handler */
void process(aggregator_operation *op) { execute_impl(*op); }
protected:
/** Place operation in mailbox, then either handle mailbox or wait for the operation
to be completed by a different thread. */
void execute_impl(aggregator_operation& op) {
aggregator_operation* res;
// ITT note: &(op.status) tag is used to cover accesses to this operation. This
// thread has created the operation, and now releases it so that the handler
// thread may handle the associated operation w/o triggering a race condition;
// thus this tag will be acquired just before the operation is handled in the
// handle_operations functor.
call_itt_notify(releasing, &(op.status));
// insert the operation in the queue
do {
// ITT may flag the following line as a race; it is a false positive:
// This is an atomic read; we don't provide itt_hide_load_word for atomics
op.my_next = res = mailbox; // NOT A RACE
} while (mailbox.compare_and_swap(&op, res) != res);
if (!res) { // first in the list; handle the operations
// ITT note: &mailbox tag covers access to the handler_busy flag, which this
// waiting handler thread will try to set before entering handle_operations.
call_itt_notify(acquired, &mailbox);
start_handle_operations();
__TBB_ASSERT(op.status, NULL);
}
else { // not first; wait for op to be ready
call_itt_notify(prepare, &(op.status));
spin_wait_while_eq(op.status, uintptr_t(aggregator_operation::agg_waiting));
itt_load_word_with_acquire(op.status);
}
}
private:
//! An atomically updated list (aka mailbox) of aggregator_operations
atomic<aggregator_operation *> mailbox;
//! Controls thread access to handle_operations
/** Behaves as boolean flag where 0=false, 1=true */
uintptr_t handler_busy;
handler_type handle_operations;
//! Trigger the handling of operations when the handler is free
void start_handle_operations() {
aggregator_operation *pending_operations;
// ITT note: &handler_busy tag covers access to mailbox as it is passed
// between active and waiting handlers. Below, the waiting handler waits until
// the active handler releases, and the waiting handler acquires &handler_busy as
// it becomes the active_handler. The release point is at the end of this
// function, when all operations in mailbox have been handled by the
// owner of this aggregator.
call_itt_notify(prepare, &handler_busy);
// get handler_busy: only one thread can possibly spin here at a time
spin_wait_until_eq(handler_busy, uintptr_t(0));
call_itt_notify(acquired, &handler_busy);
// acquire fence not necessary here due to causality rule and surrounding atomics
__TBB_store_with_release(handler_busy, uintptr_t(1));
// ITT note: &mailbox tag covers access to the handler_busy flag itself.
// Capturing the state of the mailbox signifies that handler_busy has been
// set and a new active handler will now process that list's operations.
call_itt_notify(releasing, &mailbox);
// grab pending_operations
pending_operations = mailbox.fetch_and_store(NULL);
// handle all the operations
handle_operations(pending_operations);
// release the handler
itt_store_word_with_release(handler_busy, uintptr_t(0));
}
};
//! Basic aggregator interface
class aggregator : private aggregator_ext<internal::basic_handler> {
public:
aggregator() : aggregator_ext<internal::basic_handler>(internal::basic_handler()) {}
//! BASIC INTERFACE: Enter a function for exclusive execution by the aggregator.
/** The calling thread stores the function object in a basic_operation and
places the operation in the aggregator's mailbox */
template<typename Body>
void execute(const Body& b) {
internal::basic_operation<Body> op(b);
this->execute_impl(op);
}
};
} // namespace interface6
using interface6::aggregator;
using interface6::aggregator_ext;
using interface6::aggregator_operation;
} // namespace tbb
#endif // __TBB__aggregator_H

View File

@@ -0,0 +1,47 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_aligned_space_H
#define __TBB_aligned_space_H
#include "tbb_stddef.h"
#include "tbb_machine.h"
namespace tbb {
//! Block of space aligned sufficiently to construct an array T with N elements.
/** The elements are not constructed or destroyed by this class.
@ingroup memory_allocation */
template<typename T,size_t N=1>
class aligned_space {
private:
typedef __TBB_TypeWithAlignmentAtLeastAsStrict(T) element_type;
element_type array[(sizeof(T)*N+sizeof(element_type)-1)/sizeof(element_type)];
public:
//! Pointer to beginning of array
T* begin() {return internal::punned_cast<T*>(this);}
//! Pointer to one past last element in array.
T* end() {return begin()+N;}
};
} // namespace tbb
#endif /* __TBB_aligned_space_H */

View File

@@ -0,0 +1,867 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "scheduler.h"
#include "governor.h"
#include "arena.h"
#include "itt_notify.h"
#include "semaphore.h"
#include <functional>
#if __TBB_STATISTICS_STDOUT
#include <cstdio>
#endif
namespace tbb {
namespace internal {
void arena::process( generic_scheduler& s ) {
__TBB_ASSERT( is_alive(my_guard), NULL );
__TBB_ASSERT( governor::is_set(&s), NULL );
__TBB_ASSERT( !s.my_innermost_running_task, NULL );
__TBB_ASSERT( !s.my_dispatching_task, NULL );
__TBB_ASSERT( my_num_slots != 1, NULL );
// Start search for an empty slot from the one we occupied the last time
unsigned index = s.my_arena_index < my_num_slots ? s.my_arena_index : s.my_random.get() % (my_num_slots - 1) + 1,
end = index;
__TBB_ASSERT( index != 0, "A worker cannot occupy slot 0" );
__TBB_ASSERT( index < my_num_slots, NULL );
// Find a vacant slot
for ( ;; ) {
if ( !my_slots[index].my_scheduler && as_atomic(my_slots[index].my_scheduler).compare_and_swap(&s, NULL ) == NULL )
break;
if ( ++index == my_num_slots )
index = 1;
if ( index == end ) {
// Likely this arena is already saturated
goto quit;
}
}
ITT_NOTIFY(sync_acquired, my_slots + index);
s.my_arena = this;
s.my_arena_index = index;
s.my_arena_slot = my_slots + index;
#if __TBB_TASK_PRIORITY
s.my_local_reload_epoch = *s.my_ref_reload_epoch;
__TBB_ASSERT( !s.my_offloaded_tasks, NULL );
#endif /* __TBB_TASK_PRIORITY */
s.attach_mailbox( affinity_id(index+1) );
s.my_arena_slot->hint_for_pop = index; // initial value for round-robin
#if !__TBB_FP_CONTEXT
my_cpu_ctl_env.set_env();
#endif
#if __TBB_SCHEDULER_OBSERVER
__TBB_ASSERT( !s.my_last_local_observer, "There cannot be notified local observers when entering arena" );
my_observers.notify_entry_observers( s.my_last_local_observer, /*worker=*/true );
#endif /* __TBB_SCHEDULER_OBSERVER */
atomic_update( my_limit, index + 1, std::less<unsigned>() );
for ( ;; ) {
// Try to steal a task.
// Passing reference count is technically unnecessary in this context,
// but omitting it here would add checks inside the function.
__TBB_ASSERT( is_alive(my_guard), NULL );
task* t = s.receive_or_steal_task( s.my_dummy_task->prefix().ref_count, /*return_if_no_work=*/true );
if (t) {
// A side effect of receive_or_steal_task is that my_innermost_running_task can be set.
// But for the outermost dispatch loop of a worker it has to be NULL.
s.my_innermost_running_task = NULL;
__TBB_ASSERT( !s.my_dispatching_task, NULL );
s.local_wait_for_all(*s.my_dummy_task,t);
}
__TBB_ASSERT ( __TBB_load_relaxed(s.my_arena_slot->head) == __TBB_load_relaxed(s.my_arena_slot->tail),
"Worker cannot leave arena while its task pool is not empty" );
__TBB_ASSERT( s.my_arena_slot->task_pool == EmptyTaskPool, "Empty task pool is not marked appropriately" );
// This check prevents relinquishing more than necessary workers because
// of the non-atomicity of the decision making procedure
if (num_workers_active() > my_num_workers_allotted)
break;
}
#if __TBB_SCHEDULER_OBSERVER
my_observers.notify_exit_observers( s.my_last_local_observer, /*worker=*/true );
s.my_last_local_observer = NULL;
#endif /* __TBB_SCHEDULER_OBSERVER */
#if __TBB_TASK_PRIORITY
if ( s.my_offloaded_tasks )
orphan_offloaded_tasks( s );
#endif /* __TBB_TASK_PRIORITY */
#if __TBB_STATISTICS
++s.my_counters.arena_roundtrips;
*my_slots[index].my_counters += s.my_counters;
s.my_counters.reset();
#endif /* __TBB_STATISTICS */
__TBB_store_with_release( my_slots[index].my_scheduler, (generic_scheduler*)NULL );
s.my_arena_slot = 0; // detached from slot
s.my_inbox.detach();
__TBB_ASSERT( s.my_inbox.is_idle_state(true), NULL );
__TBB_ASSERT( !s.my_innermost_running_task, NULL );
__TBB_ASSERT( !s.my_dispatching_task, NULL );
__TBB_ASSERT( is_alive(my_guard), NULL );
quit:
// In contrast to earlier versions of TBB (before 3.0 U5) now it is possible
// that arena may be temporarily left unpopulated by threads. See comments in
// arena::on_thread_leaving() for more details.
#if !__TBB_TRACK_PRIORITY_LEVEL_SATURATION
on_thread_leaving</*is_master*/false>();
#endif /* !__TBB_TRACK_PRIORITY_LEVEL_SATURATION */
}
arena::arena ( market& m, unsigned max_num_workers ) {
__TBB_ASSERT( !my_guard, "improperly allocated arena?" );
__TBB_ASSERT( sizeof(my_slots[0]) % NFS_GetLineSize()==0, "arena::slot size not multiple of cache line size" );
__TBB_ASSERT( (uintptr_t)this % NFS_GetLineSize()==0, "arena misaligned" );
#if __TBB_TASK_PRIORITY
__TBB_ASSERT( !my_reload_epoch && !my_orphaned_tasks && !my_skipped_fifo_priority, "New arena object is not zeroed" );
#endif /* __TBB_TASK_PRIORITY */
my_market = &m;
my_limit = 1;
// Two slots are mandatory: for the master, and for 1 worker (required to support starvation resistant tasks).
my_num_slots = num_slots_to_reserve(max_num_workers);
my_max_num_workers = max_num_workers;
my_references = 1; // accounts for the master
#if __TBB_TASK_PRIORITY
my_bottom_priority = my_top_priority = normalized_normal_priority;
#endif /* __TBB_TASK_PRIORITY */
my_aba_epoch = m.my_arenas_aba_epoch;
#if __TBB_SCHEDULER_OBSERVER
my_observers.my_arena = this;
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_ASSERT ( my_max_num_workers < my_num_slots, NULL );
// Construct slots. Mark internal synchronization elements for the tools.
for( unsigned i = 0; i < my_num_slots; ++i ) {
__TBB_ASSERT( !my_slots[i].my_scheduler && !my_slots[i].task_pool, NULL );
__TBB_ASSERT( !my_slots[i].task_pool_ptr, NULL );
__TBB_ASSERT( !my_slots[i].my_task_pool_size, NULL );
ITT_SYNC_CREATE(my_slots + i, SyncType_Scheduler, SyncObj_WorkerTaskPool);
mailbox(i+1).construct();
ITT_SYNC_CREATE(&mailbox(i+1), SyncType_Scheduler, SyncObj_Mailbox);
my_slots[i].hint_for_pop = i;
#if __TBB_STATISTICS
my_slots[i].my_counters = new ( NFS_Allocate(1, sizeof(statistics_counters), NULL) ) statistics_counters;
#endif /* __TBB_STATISTICS */
}
#if __TBB_TASK_PRIORITY
for ( intptr_t i = 0; i < num_priority_levels; ++i ) {
my_task_stream[i].initialize(my_num_slots);
ITT_SYNC_CREATE(my_task_stream + i, SyncType_Scheduler, SyncObj_TaskStream);
}
#else /* !__TBB_TASK_PRIORITY */
my_task_stream.initialize(my_num_slots);
ITT_SYNC_CREATE(&my_task_stream, SyncType_Scheduler, SyncObj_TaskStream);
#endif /* !__TBB_TASK_PRIORITY */
my_mandatory_concurrency = false;
#if __TBB_TASK_GROUP_CONTEXT
// Context to be used by root tasks by default (if the user has not specified one).
// The arena's context should not capture fp settings for the sake of backward compatibility.
my_default_ctx =
new ( NFS_Allocate(1, sizeof(task_group_context), NULL) ) task_group_context(task_group_context::isolated, task_group_context::default_traits);
#endif /* __TBB_TASK_GROUP_CONTEXT */
#if __TBB_FP_CONTEXT
my_default_ctx->capture_fp_settings();
#else
my_cpu_ctl_env.get_env();
#endif
}
arena& arena::allocate_arena( market& m, unsigned max_num_workers ) {
__TBB_ASSERT( sizeof(base_type) + sizeof(arena_slot) == sizeof(arena), "All arena data fields must go to arena_base" );
__TBB_ASSERT( sizeof(base_type) % NFS_GetLineSize() == 0, "arena slots area misaligned: wrong padding" );
__TBB_ASSERT( sizeof(mail_outbox) == NFS_MaxLineSize, "Mailbox padding is wrong" );
size_t n = allocation_size(max_num_workers);
unsigned char* storage = (unsigned char*)NFS_Allocate( 1, n, NULL );
// Zero all slots to indicate that they are empty
memset( storage, 0, n );
return *new( storage + num_slots_to_reserve(max_num_workers) * sizeof(mail_outbox) ) arena(m, max_num_workers);
}
void arena::free_arena () {
__TBB_ASSERT( is_alive(my_guard), NULL );
__TBB_ASSERT( !my_references, "There are threads in the dying arena" );
__TBB_ASSERT( !my_num_workers_requested && !my_num_workers_allotted, "Dying arena requests workers" );
__TBB_ASSERT( my_pool_state == SNAPSHOT_EMPTY || !my_max_num_workers, "Inconsistent state of a dying arena" );
#if !__TBB_STATISTICS_EARLY_DUMP
GATHER_STATISTIC( dump_arena_statistics() );
#endif
poison_value( my_guard );
intptr_t drained = 0;
for ( unsigned i = 0; i < my_num_slots; ++i ) {
__TBB_ASSERT( !my_slots[i].my_scheduler, "arena slot is not empty" );
#if !__TBB_TASK_ARENA
__TBB_ASSERT( my_slots[i].task_pool == EmptyTaskPool, NULL );
#else
//TODO: understand the assertion and modify
#endif
__TBB_ASSERT( my_slots[i].head == my_slots[i].tail, NULL ); // TODO: replace by is_quiescent_local_task_pool_empty
my_slots[i].free_task_pool();
#if __TBB_STATISTICS
NFS_Free( my_slots[i].my_counters );
#endif /* __TBB_STATISTICS */
drained += mailbox(i+1).drain();
}
#if __TBB_TASK_PRIORITY && TBB_USE_ASSERT
for ( intptr_t i = 0; i < num_priority_levels; ++i )
__TBB_ASSERT(my_task_stream[i].empty() && my_task_stream[i].drain()==0, "Not all enqueued tasks were executed");
#elif !__TBB_TASK_PRIORITY
__TBB_ASSERT(my_task_stream.empty() && my_task_stream.drain()==0, "Not all enqueued tasks were executed");
#endif /* !__TBB_TASK_PRIORITY */
#if __TBB_COUNT_TASK_NODES
my_market->update_task_node_count( -drained );
#endif /* __TBB_COUNT_TASK_NODES */
my_market->release();
#if __TBB_TASK_GROUP_CONTEXT
__TBB_ASSERT( my_default_ctx, "Master thread never entered the arena?" );
my_default_ctx->~task_group_context();
NFS_Free(my_default_ctx);
#endif /* __TBB_TASK_GROUP_CONTEXT */
#if __TBB_SCHEDULER_OBSERVER
if ( !my_observers.empty() )
my_observers.clear();
#endif /* __TBB_SCHEDULER_OBSERVER */
void* storage = &mailbox(my_num_slots);
__TBB_ASSERT( my_references == 0, NULL );
__TBB_ASSERT( my_pool_state == SNAPSHOT_EMPTY || !my_max_num_workers, NULL );
this->~arena();
#if TBB_USE_ASSERT > 1
memset( storage, 0, allocation_size(my_max_num_workers) );
#endif /* TBB_USE_ASSERT */
NFS_Free( storage );
}
#if __TBB_STATISTICS
void arena::dump_arena_statistics () {
statistics_counters total;
for( unsigned i = 0; i < my_num_slots; ++i ) {
#if __TBB_STATISTICS_EARLY_DUMP
generic_scheduler* s = my_slots[i].my_scheduler;
if ( s )
*my_slots[i].my_counters += s->my_counters;
#else
__TBB_ASSERT( !my_slots[i].my_scheduler, NULL );
#endif
if ( i != 0 ) {
total += *my_slots[i].my_counters;
dump_statistics( *my_slots[i].my_counters, i );
}
}
dump_statistics( *my_slots[0].my_counters, 0 );
#if __TBB_STATISTICS_STDOUT
#if !__TBB_STATISTICS_TOTALS_ONLY
printf( "----------------------------------------------\n" );
#endif
dump_statistics( total, workers_counters_total );
total += *my_slots[0].my_counters;
dump_statistics( total, arena_counters_total );
#if !__TBB_STATISTICS_TOTALS_ONLY
printf( "==============================================\n" );
#endif
#endif /* __TBB_STATISTICS_STDOUT */
}
#endif /* __TBB_STATISTICS */
#if __TBB_TASK_PRIORITY
// The method inspects a scheduler to determine:
// 1. if it has tasks that can be retrieved and executed (via the return value);
// 2. if it has any tasks at all, including those of lower priority (via tasks_present);
// 3. if it is able to work with enqueued tasks (via dequeuing_possible).
inline bool arena::may_have_tasks ( generic_scheduler* s, bool& tasks_present, bool& dequeuing_possible ) {
if ( !s
#if __TBB_TASK_ARENA
|| s->my_arena != this
#endif
) return false;
dequeuing_possible |= s->worker_outermost_level();
if ( s->my_pool_reshuffling_pending ) {
// This primary task pool is nonempty and may contain tasks at the current
// priority level. Its owner is winnowing lower priority tasks at the moment.
tasks_present = true;
return true;
}
if ( s->my_offloaded_tasks ) {
tasks_present = true;
if ( s->my_local_reload_epoch < *s->my_ref_reload_epoch ) {
// This scheduler's offload area is nonempty and may contain tasks at the
// current priority level.
return true;
}
}
return false;
}
void arena::orphan_offloaded_tasks(generic_scheduler& s) {
__TBB_ASSERT( s.my_offloaded_tasks, NULL );
GATHER_STATISTIC( ++s.my_counters.prio_orphanings );
++my_abandonment_epoch;
__TBB_ASSERT( s.my_offloaded_task_list_tail_link && !*s.my_offloaded_task_list_tail_link, NULL );
task* orphans;
do {
orphans = const_cast<task*>(my_orphaned_tasks);
*s.my_offloaded_task_list_tail_link = orphans;
} while ( as_atomic(my_orphaned_tasks).compare_and_swap(s.my_offloaded_tasks, orphans) != orphans );
s.my_offloaded_tasks = NULL;
#if TBB_USE_ASSERT
s.my_offloaded_task_list_tail_link = NULL;
#endif /* TBB_USE_ASSERT */
}
#endif /* __TBB_TASK_PRIORITY */
bool arena::is_out_of_work() {
// TODO: rework it to return at least a hint about where a task was found; better if the task itself.
for(;;) {
pool_state_t snapshot = my_pool_state;
switch( snapshot ) {
case SNAPSHOT_EMPTY:
return true;
case SNAPSHOT_FULL: {
// Use unique id for "busy" in order to avoid ABA problems.
const pool_state_t busy = pool_state_t(&busy);
// Request permission to take snapshot
if( my_pool_state.compare_and_swap( busy, SNAPSHOT_FULL )==SNAPSHOT_FULL ) {
// Got permission. Take the snapshot.
// NOTE: This is not a lock, as the state can be set to FULL at
// any moment by a thread that spawns/enqueues new task.
size_t n = my_limit;
// Make local copies of volatile parameters. Their change during
// snapshot taking procedure invalidates the attempt, and returns
// this thread into the dispatch loop.
#if __TBB_TASK_PRIORITY
intptr_t top_priority = my_top_priority;
uintptr_t reload_epoch = my_reload_epoch;
// Inspect primary task pools first
#endif /* __TBB_TASK_PRIORITY */
size_t k;
for( k=0; k<n; ++k ) {
if( my_slots[k].task_pool != EmptyTaskPool &&
__TBB_load_relaxed(my_slots[k].head) < __TBB_load_relaxed(my_slots[k].tail) )
{
// k-th primary task pool is nonempty and does contain tasks.
break;
}
if( my_pool_state!=busy )
return false; // the work was published
}
__TBB_ASSERT( k <= n, NULL );
bool work_absent = k == n;
#if __TBB_TASK_PRIORITY
// Variable tasks_present indicates presence of tasks at any priority
// level, while work_absent refers only to the current priority.
bool tasks_present = !work_absent || my_orphaned_tasks;
bool dequeuing_possible = false;
if ( work_absent ) {
// Check for the possibility that recent priority changes
// brought some tasks to the current priority level
uintptr_t abandonment_epoch = my_abandonment_epoch;
// Master thread's scheduler needs special handling as it
// may be destroyed at any moment (workers' schedulers are
// guaranteed to be alive while at least one thread is in arena).
// Have to exclude concurrency with task group state change propagation too.
// TODO: check whether it is still necessary since some pools belong to slots now
my_market->my_arenas_list_mutex.lock();
generic_scheduler *s = my_slots[0].my_scheduler;
if ( s && as_atomic(my_slots[0].my_scheduler).compare_and_swap(LockedMaster, s) == s ) { //TODO: remove need to lock
__TBB_ASSERT( my_slots[0].my_scheduler == LockedMaster && s != LockedMaster, NULL );
work_absent = !may_have_tasks( s, tasks_present, dequeuing_possible );
__TBB_store_with_release( my_slots[0].my_scheduler, s );
}
my_market->my_arenas_list_mutex.unlock();
// The following loop is subject to data races. While k-th slot's
// scheduler is being examined, corresponding worker can either
// leave to RML or migrate to another arena.
// But the races are not prevented because all of them are benign.
// First, the code relies on the fact that worker thread's scheduler
// object persists until the whole library is deinitialized.
// Second, in the worst case the races can only cause another
// round of stealing attempts to be undertaken. Introducing complex
// synchronization into this coldest part of the scheduler's control
// flow does not seem to make sense because it both is unlikely to
// ever have any observable performance effect, and will require
// additional synchronization code on the hotter paths.
for( k = 1; work_absent && k < n; ++k ) {
if( my_pool_state!=busy )
return false; // the work was published
work_absent = !may_have_tasks( my_slots[k].my_scheduler, tasks_present, dequeuing_possible );
}
// Preclude premature switching arena off because of a race in the previous loop.
work_absent = work_absent
&& !__TBB_load_with_acquire(my_orphaned_tasks)
&& abandonment_epoch == my_abandonment_epoch;
}
#endif /* __TBB_TASK_PRIORITY */
// Test and test-and-set.
if( my_pool_state==busy ) {
#if __TBB_TASK_PRIORITY
bool no_fifo_tasks = my_task_stream[top_priority].empty();
work_absent = work_absent && (!dequeuing_possible || no_fifo_tasks)
&& top_priority == my_top_priority && reload_epoch == my_reload_epoch;
#else
bool no_fifo_tasks = my_task_stream.empty();
work_absent = work_absent && no_fifo_tasks;
#endif /* __TBB_TASK_PRIORITY */
if( work_absent ) {
#if __TBB_TASK_PRIORITY
if ( top_priority > my_bottom_priority ) {
if ( my_market->lower_arena_priority(*this, top_priority - 1, reload_epoch)
&& !my_task_stream[top_priority].empty() )
{
atomic_update( my_skipped_fifo_priority, top_priority, std::less<intptr_t>());
}
}
else if ( !tasks_present && !my_orphaned_tasks && no_fifo_tasks ) {
#endif /* __TBB_TASK_PRIORITY */
// save current demand value before setting SNAPSHOT_EMPTY,
// to avoid race with advertise_new_work.
int current_demand = (int)my_max_num_workers;
if( my_pool_state.compare_and_swap( SNAPSHOT_EMPTY, busy )==busy ) {
// This thread transitioned pool to empty state, and thus is
// responsible for telling RML that there is no other work to do.
my_market->adjust_demand( *this, -current_demand );
#if __TBB_TASK_PRIORITY
// Check for the presence of enqueued tasks "lost" on some of
// priority levels because updating arena priority and switching
// arena into "populated" (FULL) state happen non-atomically.
// Imposing atomicity would require task::enqueue() to use a lock,
// which is unacceptable.
bool switch_back = false;
for ( int p = 0; p < num_priority_levels; ++p ) {
if ( !my_task_stream[p].empty() ) {
switch_back = true;
if ( p < my_bottom_priority || p > my_top_priority )
my_market->update_arena_priority(*this, p);
}
}
if ( switch_back )
advertise_new_work</*Spawned*/false>();
#endif /* __TBB_TASK_PRIORITY */
return true;
}
return false;
#if __TBB_TASK_PRIORITY
}
#endif /* __TBB_TASK_PRIORITY */
}
// Undo previous transition SNAPSHOT_FULL-->busy, unless another thread undid it.
my_pool_state.compare_and_swap( SNAPSHOT_FULL, busy );
}
}
return false;
}
default:
// Another thread is taking a snapshot.
return false;
}
}
}
#if __TBB_COUNT_TASK_NODES
intptr_t arena::workers_task_node_count() {
intptr_t result = 0;
for( unsigned i = 1; i < my_num_slots; ++i ) {
generic_scheduler* s = my_slots[i].my_scheduler;
if( s )
result += s->my_task_node_count;
}
return result;
}
#endif /* __TBB_COUNT_TASK_NODES */
void arena::enqueue_task( task& t, intptr_t prio, FastRandom &random )
{
#if __TBB_RECYCLE_TO_ENQUEUE
__TBB_ASSERT( t.state()==task::allocated || t.state()==task::to_enqueue, "attempt to enqueue task with inappropriate state" );
#else
__TBB_ASSERT( t.state()==task::allocated, "attempt to enqueue task that is not in 'allocated' state" );
#endif
t.prefix().state = task::ready;
t.prefix().extra_state |= es_task_enqueued; // enqueued task marker
#if TBB_USE_ASSERT
if( task* parent = t.parent() ) {
internal::reference_count ref_count = parent->prefix().ref_count;
__TBB_ASSERT( ref_count!=0, "attempt to enqueue task whose parent has a ref_count==0 (forgot to set_ref_count?)" );
__TBB_ASSERT( ref_count>0, "attempt to enqueue task whose parent has a ref_count<0" );
parent->prefix().extra_state |= es_ref_count_active;
}
__TBB_ASSERT(t.prefix().affinity==affinity_id(0), "affinity is ignored for enqueued tasks");
#endif /* TBB_USE_ASSERT */
#if __TBB_TASK_PRIORITY
intptr_t p = prio ? normalize_priority(priority_t(prio)) : normalized_normal_priority;
assert_priority_valid(p);
task_stream &ts = my_task_stream[p];
#else /* !__TBB_TASK_PRIORITY */
__TBB_ASSERT_EX(prio == 0, "the library is not configured to respect the task priority");
task_stream &ts = my_task_stream;
#endif /* !__TBB_TASK_PRIORITY */
ITT_NOTIFY(sync_releasing, &ts);
ts.push( &t, random );
#if __TBB_TASK_PRIORITY
if ( p != my_top_priority )
my_market->update_arena_priority( *this, p );
#endif /* __TBB_TASK_PRIORITY */
advertise_new_work< /*Spawned=*/ false >();
#if __TBB_TASK_PRIORITY
if ( p != my_top_priority )
my_market->update_arena_priority( *this, p );
#endif /* __TBB_TASK_PRIORITY */
}
#if __TBB_TASK_ARENA
struct nested_arena_context : no_copy {
generic_scheduler &my_scheduler;
scheduler_state const my_orig_state;
void *my_orig_ptr;
bool my_adjusting;
nested_arena_context(generic_scheduler *s, arena* a, bool needs_adjusting, bool as_worker = false)
: my_scheduler(*s), my_orig_state(*s), my_orig_ptr(NULL), my_adjusting(needs_adjusting) {
s->nested_arena_entry(a, *this, as_worker);
}
~nested_arena_context() {
my_scheduler.nested_arena_exit(*this);
(scheduler_state&)my_scheduler = my_orig_state; // restore arena settings
}
};
void generic_scheduler::nested_arena_entry(arena* a, nested_arena_context& c, bool as_worker) {
if( a == my_arena ) {
#if __TBB_TASK_GROUP_CONTEXT
c.my_orig_ptr = my_innermost_running_task =
new(&allocate_task(sizeof(empty_task), NULL, a->my_default_ctx)) empty_task;
#endif
return;
}
__TBB_ASSERT( is_alive(a->my_guard), NULL );
// overwrite arena settings
#if __TBB_TASK_PRIORITY
if ( my_offloaded_tasks )
my_arena->orphan_offloaded_tasks( *this );
my_ref_top_priority = &a->my_top_priority;
my_ref_reload_epoch = &a->my_reload_epoch;
my_local_reload_epoch = a->my_reload_epoch;
#endif /* __TBB_TASK_PRIORITY */
my_arena = a;
my_arena_index = 0;
my_arena_slot = my_arena->my_slots + my_arena_index;
my_inbox.detach(); // TODO: mailboxes were not designed for switching, add copy constructor?
attach_mailbox( affinity_id(my_arena_index+1) );
my_innermost_running_task = my_dispatching_task = as_worker? NULL : my_dummy_task;
#if __TBB_TASK_GROUP_CONTEXT
// save dummy's context and replace it by arena's context
c.my_orig_ptr = my_dummy_task->prefix().context;
my_dummy_task->prefix().context = a->my_default_ctx;
#endif
#if __TBB_ARENA_OBSERVER
my_last_local_observer = 0; // TODO: try optimize number of calls
my_arena->my_observers.notify_entry_observers( my_last_local_observer, /*worker=*/false );
#endif
// TODO? ITT_NOTIFY(sync_acquired, a->my_slots + index);
// TODO: it requires market to have P workers (not P-1)
// TODO: it still allows temporary oversubscription by 1 worker (due to my_max_num_workers)
// TODO: a preempted worker should be excluded from assignment to other arenas e.g. my_slack--
if( c.my_adjusting ) my_arena->my_market->adjust_demand(*my_arena, -1);
}
void generic_scheduler::nested_arena_exit(nested_arena_context& c) {
if( my_arena == c.my_orig_state.my_arena ) {
#if __TBB_TASK_GROUP_CONTEXT
free_task<small_local_task>(*(task*)c.my_orig_ptr); // TODO: use scoped_task instead?
#endif
return;
}
if( c.my_adjusting ) my_arena->my_market->adjust_demand(*my_arena, 1);
#if __TBB_ARENA_OBSERVER
my_arena->my_observers.notify_exit_observers( my_last_local_observer, /*worker=*/false );
#endif /* __TBB_SCHEDULER_OBSERVER */
#if __TBB_TASK_PRIORITY
if ( my_offloaded_tasks )
my_arena->orphan_offloaded_tasks( *this );
my_local_reload_epoch = *c.my_orig_state.my_ref_reload_epoch;
while ( as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap( NULL, this) != this )
__TBB_Yield(); // TODO: task priority can use master slot for locking while accessing the scheduler
#else
// Free the master slot. TODO: support multiple masters
__TBB_store_with_release(my_arena->my_slots[0].my_scheduler, (generic_scheduler*)NULL);
#endif
my_arena->my_exit_monitors.notify_all_relaxed(); // TODO: fix concurrent monitor to use notify_one (test MultipleMastersPart4 fails)
#if __TBB_TASK_GROUP_CONTEXT
// restore context of dummy task
my_dummy_task->prefix().context = (task_group_context*)c.my_orig_ptr;
#endif
}
void generic_scheduler::wait_until_empty() {
my_dummy_task->prefix().ref_count++; // prevents exit from local_wait_for_all when local work is done enforcing the stealing
while( my_arena->my_pool_state != arena::SNAPSHOT_EMPTY )
local_wait_for_all(*my_dummy_task, NULL);
my_dummy_task->prefix().ref_count--;
}
#endif /* __TBB_TASK_ARENA */
} // namespace internal
} // namespace tbb
#if __TBB_TASK_ARENA
#include "scheduler_utility.h"
namespace tbb {
namespace interface7 {
namespace internal {
void task_arena_base::internal_initialize( ) {
__TBB_ASSERT( my_master_slots <= 1, "Number of slots reserved for master can be only [0,1]");
if( my_master_slots > 1 ) my_master_slots = 1; // TODO: make more masters
if( my_max_concurrency < 1 )
my_max_concurrency = (int)governor::default_num_threads();
// TODO: reimplement in an efficient way. We need a scheduler instance in this thread
// but the scheduler is only required for task allocation and fifo random seeds until
// master wants to join the arena. (Idea - to create a restricted specialization)
// It is excessive to create an implicit arena for master here anyway. But scheduler
// instance implies master thread to be always connected with arena.
// browse recursively into init_scheduler and arena::process for details
if( !governor::local_scheduler_if_initialized() )
governor::init_scheduler( (unsigned)my_max_concurrency - my_master_slots + 1/*TODO: address in market instead*/, 0, true );
// TODO: we will need to introduce a mechanism for global settings, including stack size, used by all arenas
arena* new_arena = &market::create_arena( my_max_concurrency - my_master_slots/*it's +1 slot for num_masters=0*/, ThreadStackSize );
if(as_atomic(my_arena).compare_and_swap(new_arena, NULL) != NULL) { // there is a race possible on my_initialized
__TBB_ASSERT(my_arena, NULL); // other thread was the first
new_arena->on_thread_leaving</*is_master*/true>(); // deallocate new arena
}
#if __TBB_TASK_GROUP_CONTEXT
else {
my_context = new_arena->my_default_ctx;
my_context->my_version_and_traits |= my_version_and_traits & exact_exception_flag;
}
#endif
}
void task_arena_base::internal_terminate( ) {
if( my_arena ) {// task_arena was initialized
#if __TBB_STATISTICS_EARLY_DUMP
GATHER_STATISTIC( my_arena->dump_arena_statistics() );
#endif
my_arena->on_thread_leaving</*is_master*/true>();
my_arena = 0;
#if __TBB_TASK_GROUP_CONTEXT
my_context = 0;
#endif
}
}
void task_arena_base::internal_enqueue( task& t, intptr_t prio ) const {
__TBB_ASSERT(my_arena, NULL);
generic_scheduler* s = governor::local_scheduler_if_initialized();
__TBB_ASSERT(s, "Scheduler is not initialized"); // we allocated a task so can expect the scheduler
#if __TBB_TASK_GROUP_CONTEXT
__TBB_ASSERT(my_arena->my_default_ctx == t.prefix().context, NULL);
__TBB_ASSERT(!my_arena->my_default_ctx->is_group_execution_cancelled(), // TODO: any better idea?
"The task will not be executed because default task_group_context of task_arena is cancelled. Has previously enqueued task thrown an exception?");
#endif
my_arena->enqueue_task( t, prio, s->my_random );
}
class delegated_task : public task {
internal::delegate_base & my_delegate;
concurrent_monitor & my_monitor;
task * my_root;
/*override*/ task* execute() {
generic_scheduler& s = *(generic_scheduler*)prefix().owner;
__TBB_ASSERT(s.worker_outermost_level() || s.master_outermost_level(), "expected to be enqueued and received on the outermost level");
// but this task can mimics outermost level, detect it
if( s.master_outermost_level() && s.my_dummy_task->state() == task::executing ) {
#if TBB_USE_EXCEPTIONS
// RTTI is available, check whether the cast is valid
__TBB_ASSERT(dynamic_cast<delegated_task*>(s.my_dummy_task), 0);
#endif
set_ref_count(1); // required by the semantics of recycle_to_enqueue()
recycle_to_enqueue();
return NULL;
}
struct outermost_context : internal::no_copy {
delegated_task * t;
generic_scheduler & s;
task * orig_dummy;
task_group_context * orig_ctx;
outermost_context(delegated_task *_t, generic_scheduler &_s) : t(_t), s(_s) {
orig_dummy = s.my_dummy_task;
#if __TBB_TASK_GROUP_CONTEXT
orig_ctx = t->prefix().context;
t->prefix().context = s.my_arena->my_default_ctx;
#endif
s.my_dummy_task = t; // mimics outermost master
__TBB_ASSERT(s.my_innermost_running_task == t, NULL);
}
~outermost_context() {
s.my_dummy_task = orig_dummy;
#if TBB_USE_EXCEPTIONS
// restore context for sake of registering potential exception
t->prefix().context = orig_ctx;
#endif
}
} scope(this, s);
my_delegate();
return NULL;
}
~delegated_task() {
// potential exception was already registered. It must happen before the notification
__TBB_ASSERT(my_root->ref_count()==2, NULL);
__TBB_store_with_release(my_root->prefix().ref_count, 1); // must precede the wakeup
my_monitor.notify_relaxed(*this);
}
public:
delegated_task( internal::delegate_base & d, concurrent_monitor & s, task * t )
: my_delegate(d), my_monitor(s), my_root(t) {}
// predicate for concurrent_monitor notification
bool operator()(uintptr_t ctx) const { return (void*)ctx == (void*)&my_delegate; }
};
void task_arena_base::internal_execute( internal::delegate_base& d) const {
__TBB_ASSERT(my_arena, NULL);
generic_scheduler* s = governor::local_scheduler();
__TBB_ASSERT(s, "Scheduler is not initialized");
// TODO: is it safe to assign slot to a scheduler which is not yet switched?
// TODO TEMP: one master, make more masters
if( s->my_arena == my_arena || (!__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler)
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL ) == NULL) ) {
cpu_ctl_env_helper cpu_ctl_helper;
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(my_context) );
#if TBB_USE_EXCEPTIONS
try {
#endif
//TODO: replace dummy tasks for workers as well to avoid using of the_dummy_context
nested_arena_context scope(s, my_arena, !my_master_slots);
d();
#if TBB_USE_EXCEPTIONS
} catch(...) {
cpu_ctl_helper.restore_default(); // TODO: is it needed on Windows?
if( my_version_and_traits & exact_exception_flag ) throw;
else {
task_group_context exception_container( task_group_context::isolated,
task_group_context::default_traits & ~task_group_context::exact_exception );
exception_container.register_pending_exception();
__TBB_ASSERT(exception_container.my_exception, NULL);
exception_container.my_exception->throw_self();
}
}
#endif
} else {
concurrent_monitor::thread_context waiter;
#if __TBB_TASK_GROUP_CONTEXT
task_group_context exec_context( task_group_context::isolated, my_version_and_traits & exact_exception_flag );
#if __TBB_FP_CONTEXT
exec_context.copy_fp_settings( *my_context );
#endif
#endif
auto_empty_task root(__TBB_CONTEXT_ARG(s, &exec_context));
root.prefix().ref_count = 2;
my_arena->enqueue_task( *new( task::allocate_root(__TBB_CONTEXT_ARG1(exec_context)) )
delegated_task(d, my_arena->my_exit_monitors, &root),
0, s->my_random ); // TODO: priority?
do {
my_arena->my_exit_monitors.prepare_wait(waiter, (uintptr_t)&d);
if( __TBB_load_with_acquire(root.prefix().ref_count) < 2 ) {
my_arena->my_exit_monitors.cancel_wait(waiter);
break;
}
else if( !__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler) // TODO: refactor into a function?
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL ) == NULL ) {
my_arena->my_exit_monitors.cancel_wait(waiter);
nested_arena_context scope(s, my_arena, !my_master_slots);
s->local_wait_for_all(root, NULL);
#if TBB_USE_EXCEPTIONS
__TBB_ASSERT( !exec_context.my_exception, NULL ); // exception can be thrown above, not deferred
#endif
__TBB_ASSERT( root.prefix().ref_count == 0, NULL );
break;
} else {
my_arena->my_exit_monitors.commit_wait(waiter);
}
} while( __TBB_load_with_acquire(root.prefix().ref_count) == 2 );
#if TBB_USE_EXCEPTIONS
// process possible exception
if( task_group_context::exception_container_type *pe = exec_context.my_exception )
pe->throw_self();
#endif
}
}
// this wait task is a temporary approach to wait for arena emptiness for masters without slots
// TODO: it will be rather reworked for one source of notification from is_out_of_work
class wait_task : public task {
binary_semaphore & my_signal;
/*override*/ task* execute() {
generic_scheduler* s = governor::local_scheduler_if_initialized();
__TBB_ASSERT( s, NULL );
if( s->my_arena_index && s->worker_outermost_level() ) {// on outermost level of workers only
s->local_wait_for_all( *s->my_dummy_task, NULL ); // run remaining tasks
} else s->my_arena->is_out_of_work(); // avoids starvation of internal_wait: issuing this task makes arena full
my_signal.V();
return NULL;
}
public:
wait_task ( binary_semaphore & sema ) : my_signal(sema) {}
};
void task_arena_base::internal_wait() const {
__TBB_ASSERT(my_arena, NULL);
generic_scheduler* s = governor::local_scheduler();
__TBB_ASSERT(s, "Scheduler is not initialized");
__TBB_ASSERT(s->my_arena != my_arena || s->my_arena_index == 0, "task_arena::wait_until_empty() is not supported within a worker context" );
if( s->my_arena == my_arena ) {
//unsupported, but try do something for outermost master
__TBB_ASSERT(s->master_outermost_level(), "unsupported");
if( !s->my_arena_index )
while( my_arena->num_workers_active() )
s->wait_until_empty();
} else for(;;) {
while( my_arena->my_pool_state != arena::SNAPSHOT_EMPTY ) {
if( !__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler) // TODO TEMP: one master, make more masters
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL) == NULL ) {
nested_arena_context a(s, my_arena, !my_master_slots, true);
s->wait_until_empty();
} else {
binary_semaphore waiter; // TODO: replace by a single event notification from is_out_of_work
internal_enqueue( *new( task::allocate_root(__TBB_CONTEXT_ARG1(*my_context)) ) wait_task(waiter), 0 ); // TODO: priority?
waiter.P(); // TODO: concurrent_monitor
}
}
if( !my_arena->num_workers_active() && !my_arena->my_slots[0].my_scheduler) // no activity
break; // spin until workers active but avoid spinning in a worker
__TBB_Yield(); // wait until workers and master leave
}
}
/*static*/ int task_arena_base::internal_current_slot() {
generic_scheduler* s = governor::local_scheduler_if_initialized();
return s? int(s->my_arena_index) : -1;
}
} // tbb::interfaceX::internal
} // tbb::interfaceX
} // tbb
#endif /* __TBB_TASK_ARENA */

View File

@@ -0,0 +1,396 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_arena_H
#define _TBB_arena_H
#include "tbb/tbb_stddef.h"
#include "tbb/atomic.h"
#include "tbb/tbb_machine.h"
#include "scheduler_common.h"
#include "intrusive_list.h"
#include "task_stream.h"
#include "../rml/include/rml_tbb.h"
#include "mailbox.h"
#include "observer_proxy.h"
#include "market.h"
#include "governor.h"
#if __TBB_TASK_ARENA
#include "concurrent_monitor.h"
#endif
namespace tbb {
class task_group_context;
class allocate_root_with_context_proxy;
namespace internal {
//! arena data except the array of slots
/** Separated in order to simplify padding.
Intrusive list node base class is used by market to form a list of arenas. **/
struct arena_base : padded<intrusive_list_node> {
//! Number of workers that have been marked out by the resource manager to service the arena
unsigned my_num_workers_allotted; // heavy use in stealing loop
//! References of the arena
/** Counts workers and master references separately. Bit 0 indicates reference from implicit
master or explicit task_arena; the next bits contain number of workers servicing the arena.*/
atomic<unsigned> my_references; // heavy use in stealing loop
#if __TBB_TASK_PRIORITY
//! Highest priority of recently spawned or enqueued tasks.
volatile intptr_t my_top_priority; // heavy use in stealing loop
//! Maximal currently busy slot.
atomic<unsigned> my_limit; // heavy use in stealing loop
//! Task pool for the tasks scheduled via task::enqueue() method
/** Such scheduling guarantees eventual execution even if
- new tasks are constantly coming (by extracting scheduled tasks in
relaxed FIFO order);
- the enqueuing thread does not call any of wait_for_all methods. **/
task_stream my_task_stream[num_priority_levels]; // heavy use in stealing loop
#else /* !__TBB_TASK_PRIORITY */
//! Task pool for the tasks scheduled via task::enqueue() method
/** Such scheduling guarantees eventual execution even if
- new tasks are constantly coming (by extracting scheduled tasks in
relaxed FIFO order);
- the enqueuing thread does not call any of wait_for_all methods. **/
task_stream my_task_stream; // heavy use in stealing loop
//! Maximal currently busy slot.
atomic<unsigned> my_limit; // heavy use in stealing loop
#endif /* !__TBB_TASK_PRIORITY */
//! Number of workers that are currently requested from the resource manager
int my_num_workers_requested;
//! Number of slots in the arena
unsigned my_num_slots;
//! Number of workers requested by the master thread owning the arena
unsigned my_max_num_workers;
//! Market owning this arena
market* my_market;
//! ABA prevention marker
uintptr_t my_aba_epoch;
#if !__TBB_FP_CONTEXT
//! FPU control settings of arena's master thread captured at the moment of arena instantiation.
__TBB_cpu_ctl_env_t my_cpu_ctl_env;
#endif
#if __TBB_TRACK_PRIORITY_LEVEL_SATURATION
int my_num_workers_present;
#endif /* __TBB_TRACK_PRIORITY_LEVEL_SATURATION */
//! Current task pool state and estimate of available tasks amount.
/** The estimate is either 0 (SNAPSHOT_EMPTY) or infinity (SNAPSHOT_FULL).
Special state is "busy" (any other unsigned value).
Note that the implementation of arena::is_busy_or_empty() requires
my_pool_state to be unsigned. */
tbb::atomic<uintptr_t> my_pool_state;
#if __TBB_TASK_GROUP_CONTEXT
//! Default task group context.
/** Used by root tasks allocated directly by the master thread (not from inside
a TBB task) without explicit context specification. **/
task_group_context* my_default_ctx;
#endif /* __TBB_TASK_GROUP_CONTEXT */
#if __TBB_SCHEDULER_OBSERVER
//! List of local observers attached to this arena.
observer_list my_observers;
#endif /* __TBB_SCHEDULER_OBSERVER */
#if __TBB_TASK_PRIORITY
//! Lowest normalized priority of available spawned or enqueued tasks.
intptr_t my_bottom_priority;
//! Tracks events that may bring tasks in offload areas to the top priority level.
/** Incremented when arena top priority changes or a task group priority
is elevated to the current arena's top level. **/
uintptr_t my_reload_epoch;
//! List of offloaded tasks abandoned by workers revoked by the market
task* my_orphaned_tasks;
//! Counter used to track the occurrence of recent orphaning and re-sharing operations.
tbb::atomic<uintptr_t> my_abandonment_epoch;
//! Highest priority level containing enqueued tasks
/** It being greater than 0 means that high priority enqueued tasks had to be
bypassed because all workers were blocked in nested dispatch loops and
were unable to progress at then current priority level. **/
tbb::atomic<intptr_t> my_skipped_fifo_priority;
#endif /* !__TBB_TASK_PRIORITY */
//! Indicates if there is an oversubscribing worker created to service enqueued tasks.
bool my_mandatory_concurrency;
#if __TBB_TASK_ARENA
//! exit notifications after arena slot is released
concurrent_monitor my_exit_monitors;
#endif
#if TBB_USE_ASSERT
//! Used to trap accesses to the object after its destruction.
uintptr_t my_guard;
#endif /* TBB_USE_ASSERT */
}; // struct arena_base
class arena
#if (__GNUC__<4 || __GNUC__==4 && __GNUC_MINOR__==0) && !__INTEL_COMPILER
: public padded<arena_base>
#else
: private padded<arena_base>
#endif
{
private:
friend class generic_scheduler;
template<typename SchedulerTraits> friend class custom_scheduler;
friend class governor;
friend class task_scheduler_observer_v3;
friend class market;
friend class tbb::task;
friend class tbb::task_group_context;
friend class allocate_root_with_context_proxy;
friend class intrusive_list<arena>;
friend class interface7::internal::task_arena_base; // declared in scheduler_common.h
friend class interface7::internal::delegated_task;
friend class interface7::internal::wait_task;
typedef padded<arena_base> base_type;
//! Constructor
arena ( market&, unsigned max_num_workers );
//! Allocate an instance of arena.
static arena& allocate_arena( market&, unsigned max_num_workers );
static int unsigned num_slots_to_reserve ( unsigned max_num_workers ) {
return max(2u, max_num_workers + 1);
}
static int allocation_size ( unsigned max_num_workers ) {
return sizeof(base_type) + num_slots_to_reserve(max_num_workers) * (sizeof(mail_outbox) + sizeof(arena_slot));
}
#if __TBB_TASK_GROUP_CONTEXT
//! Finds all contexts affected by the state change and propagates the new state to them.
/** The propagation is relayed to the market because tasks created by one
master thread can be passed to and executed by other masters. This means
that context trees can span several arenas at once and thus state change
propagation cannot be generally localized to one arena only. **/
template <typename T>
bool propagate_task_group_state ( T task_group_context::*mptr_state, task_group_context& src, T new_state );
#endif /* __TBB_TASK_GROUP_CONTEXT */
//! Get reference to mailbox corresponding to given affinity_id.
mail_outbox& mailbox( affinity_id id ) {
__TBB_ASSERT( 0<id, "affinity id must be positive integer" );
__TBB_ASSERT( id <= my_num_slots, "affinity id out of bounds" );
return ((mail_outbox*)this)[-(int)id];
}
//! Completes arena shutdown, destructs and deallocates it.
void free_arena ();
typedef uintptr_t pool_state_t;
//! No tasks to steal since last snapshot was taken
static const pool_state_t SNAPSHOT_EMPTY = 0;
//! At least one task has been offered for stealing since the last snapshot started
static const pool_state_t SNAPSHOT_FULL = pool_state_t(-1);
//! No tasks to steal or snapshot is being taken.
static bool is_busy_or_empty( pool_state_t s ) { return s < SNAPSHOT_FULL; }
//! The number of workers active in the arena.
unsigned num_workers_active( ) {
return my_references >> 1;
}
//! If necessary, raise a flag that there is new job in arena.
template<bool Spawned> void advertise_new_work();
//! Check if there is job anywhere in arena.
/** Return true if no job or if arena is being cleaned up. */
bool is_out_of_work();
//! enqueue a task into starvation-resistance queue
void enqueue_task( task&, intptr_t, FastRandom & );
//! Registers the worker with the arena and enters TBB scheduler dispatch loop
void process( generic_scheduler& );
//! Notification that worker or master leaves its arena
template<bool is_master>
inline void on_thread_leaving ( );
#if __TBB_STATISTICS
//! Outputs internal statistics accumulated by the arena
void dump_arena_statistics ();
#endif /* __TBB_STATISTICS */
#if __TBB_TASK_PRIORITY
//! Check if recent priority changes may bring some tasks to the current priority level soon
/** /param tasks_present indicates presence of tasks at any priority level. **/
inline bool may_have_tasks ( generic_scheduler*, bool& tasks_present, bool& dequeuing_possible );
//! Puts offloaded tasks into global list of orphaned tasks
void orphan_offloaded_tasks ( generic_scheduler& s );
#endif /* __TBB_TASK_PRIORITY */
#if __TBB_COUNT_TASK_NODES
//! Returns the number of task objects "living" in worker threads
intptr_t workers_task_node_count();
#endif
/** Must be the last data field */
arena_slot my_slots[1];
}; // class arena
template<bool is_master>
inline void arena::on_thread_leaving ( ) {
//
// Implementation of arena destruction synchronization logic contained various
// bugs/flaws at the different stages of its evolution, so below is a detailed
// description of the issues taken into consideration in the framework of the
// current design.
//
// In case of using fire-and-forget tasks (scheduled via task::enqueue())
// master thread is allowed to leave its arena before all its work is executed,
// and market may temporarily revoke all workers from this arena. Since revoked
// workers never attempt to reset arena state to EMPTY and cancel its request
// to RML for threads, the arena object is destroyed only when both the last
// thread is leaving it and arena's state is EMPTY (that is its master thread
// left and it does not contain any work).
//
// A worker that checks for work presence and transitions arena to the EMPTY
// state (in snapshot taking procedure arena::is_out_of_work()) updates
// arena::my_pool_state first and only then arena::my_num_workers_requested.
// So the check for work absence must be done against the latter field.
//
// In a time window between decrementing the active threads count and checking
// if there is an outstanding request for workers. New worker thread may arrive,
// finish remaining work, set arena state to empty, and leave decrementing its
// refcount and destroying. Then the current thread will destroy the arena
// the second time. To preclude it a local copy of the outstanding request
// value can be stored before decrementing active threads count.
//
// But this technique may cause two other problem. When the stored request is
// zero, it is possible that arena still has threads and they can generate new
// tasks and thus re-establish non-zero requests. Then all the threads can be
// revoked (as described above) leaving this thread the last one, and causing
// it to destroy non-empty arena.
//
// The other problem takes place when the stored request is non-zero. Another
// thread may complete the work, set arena state to empty, and leave without
// arena destruction before this thread decrements the refcount. This thread
// cannot destroy the arena either. Thus the arena may be "orphaned".
//
// In both cases we cannot dereference arena pointer after the refcount is
// decremented, as our arena may already be destroyed.
//
// If this is the master thread, market can be concurrently destroyed.
// In case of workers market's liveness is ensured by the RML connection
// rundown protocol, according to which the client (i.e. the market) lives
// until RML server notifies it about connection termination, and this
// notification is fired only after all workers return into RML.
//
// Thus if we decremented refcount to zero we ask the market to check arena
// state (including the fact if it is alive) under the lock.
//
uintptr_t aba_epoch = my_aba_epoch;
market* m = my_market;
__TBB_ASSERT(my_references > int(!is_master), "broken arena reference counter");
if ( (my_references -= is_master? 1:2 ) == 0 ) // worker's counter starts from bit 1
market::try_destroy_arena( m, this, aba_epoch, is_master );
}
template<bool Spawned> void arena::advertise_new_work() {
if( !Spawned ) { // i.e. the work was enqueued
if( my_max_num_workers==0 ) {
my_max_num_workers = 1;
__TBB_ASSERT(!my_mandatory_concurrency, "");
my_mandatory_concurrency = true;
__TBB_ASSERT(!num_workers_active(), "");
my_pool_state = SNAPSHOT_FULL;
my_market->adjust_demand( *this, 1 );
return;
}
// Local memory fence is required to avoid missed wakeups; see the comment below.
// Starvation resistant tasks require mandatory concurrency, so missed wakeups are unacceptable.
atomic_fence();
}
// Double-check idiom that, in case of spawning, is deliberately sloppy about memory fences.
// Technically, to avoid missed wakeups, there should be a full memory fence between the point we
// released the task pool (i.e. spawned task) and read the arena's state. However, adding such a
// fence might hurt overall performance more than it helps, because the fence would be executed
// on every task pool release, even when stealing does not occur. Since TBB allows parallelism,
// but never promises parallelism, the missed wakeup is not a correctness problem.
pool_state_t snapshot = my_pool_state;
if( is_busy_or_empty(snapshot) ) {
// Attempt to mark as full. The compare_and_swap below is a little unusual because the
// result is compared to a value that can be different than the comparand argument.
if( my_pool_state.compare_and_swap( SNAPSHOT_FULL, snapshot )==SNAPSHOT_EMPTY ) {
if( snapshot!=SNAPSHOT_EMPTY ) {
// This thread read "busy" into snapshot, and then another thread transitioned
// my_pool_state to "empty" in the meantime, which caused the compare_and_swap above
// to fail. Attempt to transition my_pool_state from "empty" to "full".
if( my_pool_state.compare_and_swap( SNAPSHOT_FULL, SNAPSHOT_EMPTY )!=SNAPSHOT_EMPTY ) {
// Some other thread transitioned my_pool_state from "empty", and hence became
// responsible for waking up workers.
return;
}
}
// This thread transitioned pool from empty to full state, and thus is responsible for
// telling RML that there is work to do.
if( Spawned ) {
if( my_mandatory_concurrency ) {
__TBB_ASSERT(my_max_num_workers==1, "");
__TBB_ASSERT(!governor::local_scheduler()->is_worker(), "");
// There was deliberate oversubscription on 1 core for sake of starvation-resistant tasks.
// Now a single active thread (must be the master) supposedly starts a new parallel region
// with relaxed sequential semantics, and oversubscription should be avoided.
// Demand for workers has been decreased to 0 during SNAPSHOT_EMPTY, so just keep it.
my_max_num_workers = 0;
my_mandatory_concurrency = false;
return;
}
}
my_market->adjust_demand( *this, my_max_num_workers );
}
}
}
} // namespace internal
} // namespace tbb
#endif /* _TBB_arena_H */

View File

@@ -0,0 +1,556 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_atomic_H
#define __TBB_atomic_H
#include <cstddef>
#if _MSC_VER
#define __TBB_LONG_LONG __int64
#else
#define __TBB_LONG_LONG long long
#endif /* _MSC_VER */
#include "tbb_machine.h"
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
// Workaround for overzealous compiler warnings
#pragma warning (push)
#pragma warning (disable: 4244 4267 4512)
#endif
namespace tbb {
//! Specifies memory semantics.
enum memory_semantics {
//! Sequential consistency
full_fence,
//! Acquire
acquire,
//! Release
release,
//! No ordering
relaxed
};
//! @cond INTERNAL
namespace internal {
#if __TBB_ATTRIBUTE_ALIGNED_PRESENT
#define __TBB_DECL_ATOMIC_FIELD(t,f,a) t f __attribute__ ((aligned(a)));
#elif __TBB_DECLSPEC_ALIGN_PRESENT
#define __TBB_DECL_ATOMIC_FIELD(t,f,a) __declspec(align(a)) t f;
#else
#error Do not know syntax for forcing alignment.
#endif
template<size_t S>
struct atomic_rep; // Primary template declared, but never defined.
template<>
struct atomic_rep<1> { // Specialization
typedef int8_t word;
};
template<>
struct atomic_rep<2> { // Specialization
typedef int16_t word;
};
template<>
struct atomic_rep<4> { // Specialization
#if _MSC_VER && !_WIN64
// Work-around that avoids spurious /Wp64 warnings
typedef intptr_t word;
#else
typedef int32_t word;
#endif
};
#if __TBB_64BIT_ATOMICS
template<>
struct atomic_rep<8> { // Specialization
typedef int64_t word;
};
#endif
template<typename value_type, size_t size>
struct aligned_storage;
//the specializations are needed to please MSVC syntax of __declspec(align()) which accept _literal_ constants only
#if __TBB_ATOMIC_CTORS
#define ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(S) \
template<typename value_type> \
struct aligned_storage<value_type,S> { \
__TBB_DECL_ATOMIC_FIELD(value_type,my_value,S) \
aligned_storage() = default ; \
constexpr aligned_storage(value_type value):my_value(value){} \
}; \
#else
#define ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(S) \
template<typename value_type> \
struct aligned_storage<value_type,S> { \
__TBB_DECL_ATOMIC_FIELD(value_type,my_value,S) \
}; \
#endif
template<typename value_type>
struct aligned_storage<value_type,1> {
value_type my_value;
#if __TBB_ATOMIC_CTORS
aligned_storage() = default ;
constexpr aligned_storage(value_type value):my_value(value){}
#endif
};
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(2)
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(4)
#if __TBB_64BIT_ATOMICS
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(8)
#endif
template<size_t Size, memory_semantics M>
struct atomic_traits; // Primary template declared, but not defined.
#define __TBB_DECL_FENCED_ATOMIC_PRIMITIVES(S,M) \
template<> struct atomic_traits<S,M> { \
typedef atomic_rep<S>::word word; \
inline static word compare_and_swap( volatile void* location, word new_value, word comparand ) { \
return __TBB_machine_cmpswp##S##M(location,new_value,comparand); \
} \
inline static word fetch_and_add( volatile void* location, word addend ) { \
return __TBB_machine_fetchadd##S##M(location,addend); \
} \
inline static word fetch_and_store( volatile void* location, word value ) { \
return __TBB_machine_fetchstore##S##M(location,value); \
} \
};
#define __TBB_DECL_ATOMIC_PRIMITIVES(S) \
template<memory_semantics M> \
struct atomic_traits<S,M> { \
typedef atomic_rep<S>::word word; \
inline static word compare_and_swap( volatile void* location, word new_value, word comparand ) { \
return __TBB_machine_cmpswp##S(location,new_value,comparand); \
} \
inline static word fetch_and_add( volatile void* location, word addend ) { \
return __TBB_machine_fetchadd##S(location,addend); \
} \
inline static word fetch_and_store( volatile void* location, word value ) { \
return __TBB_machine_fetchstore##S(location,value); \
} \
};
template<memory_semantics M>
struct atomic_load_store_traits; // Primary template declaration
#define __TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(M) \
template<> struct atomic_load_store_traits<M> { \
template <typename T> \
inline static T load( const volatile T& location ) { \
return __TBB_load_##M( location ); \
} \
template <typename T> \
inline static void store( volatile T& location, T value ) { \
__TBB_store_##M( location, value ); \
} \
}
#if __TBB_USE_FENCED_ATOMICS
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,full_fence)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,full_fence)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,full_fence)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,acquire)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,acquire)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,acquire)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,release)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,release)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,release)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,relaxed)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,relaxed)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,relaxed)
#if __TBB_64BIT_ATOMICS
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,full_fence)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,acquire)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,release)
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,relaxed)
#endif
#else /* !__TBB_USE_FENCED_ATOMICS */
__TBB_DECL_ATOMIC_PRIMITIVES(1)
__TBB_DECL_ATOMIC_PRIMITIVES(2)
__TBB_DECL_ATOMIC_PRIMITIVES(4)
#if __TBB_64BIT_ATOMICS
__TBB_DECL_ATOMIC_PRIMITIVES(8)
#endif
#endif /* !__TBB_USE_FENCED_ATOMICS */
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(full_fence);
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(acquire);
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(release);
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(relaxed);
//! Additive inverse of 1 for type T.
/** Various compilers issue various warnings if -1 is used with various integer types.
The baroque expression below avoids all the warnings (we hope). */
#define __TBB_MINUS_ONE(T) (T(T(0)-T(1)))
//! Base class that provides basic functionality for atomic<T> without fetch_and_add.
/** Works for any type T that has the same size as an integral type, has a trivial constructor/destructor,
and can be copied/compared by memcpy/memcmp. */
template<typename T>
struct atomic_impl {
protected:
aligned_storage<T,sizeof(T)> my_storage;
private:
//TODO: rechecks on recent versions of gcc if union is still the _only_ way to do a conversion without warnings
//! Union type used to convert type T to underlying integral type.
template<typename value_type>
union converter {
typedef typename atomic_rep<sizeof(value_type)>::word bits_type;
converter(){}
converter(value_type a_value) : value(a_value) {}
value_type value;
bits_type bits;
};
template<typename value_t>
static typename converter<value_t>::bits_type to_bits(value_t value){
return converter<value_t>(value).bits;
}
template<typename value_t>
static value_t to_value(typename converter<value_t>::bits_type bits){
converter<value_t> u;
u.bits = bits;
return u.value;
}
template<typename value_t>
union ptr_converter; //Primary template declared, but never defined.
template<typename value_t>
union ptr_converter<value_t *> {
ptr_converter(){}
ptr_converter(value_t* a_value) : value(a_value) {}
value_t* value;
uintptr_t bits;
};
//TODO: check if making to_bits accepting reference (thus unifying it with to_bits_ref)
//does not hurt performance
template<typename value_t>
static typename converter<value_t>::bits_type & to_bits_ref(value_t& value){
//TODO: this #ifdef is temporary workaround, as union conversion seems to fail
//on suncc for 64 bit types for 32 bit target
#if !__SUNPRO_CC
return *(typename converter<value_t>::bits_type*)ptr_converter<value_t*>(&value).bits;
#else
return *(typename converter<value_t>::bits_type*)(&value);
#endif
}
public:
typedef T value_type;
#if __TBB_ATOMIC_CTORS
atomic_impl() = default ;
constexpr atomic_impl(value_type value):my_storage(value){}
#endif
template<memory_semantics M>
value_type fetch_and_store( value_type value ) {
return to_value<value_type>(
internal::atomic_traits<sizeof(value_type),M>::fetch_and_store( &my_storage.my_value, to_bits(value) )
);
}
value_type fetch_and_store( value_type value ) {
return fetch_and_store<full_fence>(value);
}
template<memory_semantics M>
value_type compare_and_swap( value_type value, value_type comparand ) {
return to_value<value_type>(
internal::atomic_traits<sizeof(value_type),M>::compare_and_swap( &my_storage.my_value, to_bits(value), to_bits(comparand) )
);
}
value_type compare_and_swap( value_type value, value_type comparand ) {
return compare_and_swap<full_fence>(value,comparand);
}
operator value_type() const volatile { // volatile qualifier here for backwards compatibility
return to_value<value_type>(
__TBB_load_with_acquire( to_bits_ref(my_storage.my_value) )
);
}
template<memory_semantics M>
value_type load () const {
return to_value<value_type>(
internal::atomic_load_store_traits<M>::load( to_bits_ref(my_storage.my_value) )
);
}
value_type load () const {
return load<acquire>();
}
template<memory_semantics M>
void store ( value_type value ) {
internal::atomic_load_store_traits<M>::store( to_bits_ref(my_storage.my_value), to_bits(value));
}
void store ( value_type value ) {
store<release>( value );
}
protected:
value_type store_with_release( value_type rhs ) {
//TODO: unify with store<release>
__TBB_store_with_release( to_bits_ref(my_storage.my_value), to_bits(rhs) );
return rhs;
}
};
//! Base class that provides basic functionality for atomic<T> with fetch_and_add.
/** I is the underlying type.
D is the difference type.
StepType should be char if I is an integral type, and T if I is a T*. */
template<typename I, typename D, typename StepType>
struct atomic_impl_with_arithmetic: atomic_impl<I> {
public:
typedef I value_type;
#if __TBB_ATOMIC_CTORS
atomic_impl_with_arithmetic() = default ;
constexpr atomic_impl_with_arithmetic(value_type value): atomic_impl<I>(value){}
#endif
template<memory_semantics M>
value_type fetch_and_add( D addend ) {
return value_type(internal::atomic_traits<sizeof(value_type),M>::fetch_and_add( &this->my_storage.my_value, addend*sizeof(StepType) ));
}
value_type fetch_and_add( D addend ) {
return fetch_and_add<full_fence>(addend);
}
template<memory_semantics M>
value_type fetch_and_increment() {
return fetch_and_add<M>(1);
}
value_type fetch_and_increment() {
return fetch_and_add(1);
}
template<memory_semantics M>
value_type fetch_and_decrement() {
return fetch_and_add<M>(__TBB_MINUS_ONE(D));
}
value_type fetch_and_decrement() {
return fetch_and_add(__TBB_MINUS_ONE(D));
}
public:
value_type operator+=( D value ) {
return fetch_and_add(value)+value;
}
value_type operator-=( D value ) {
// Additive inverse of value computed using binary minus,
// instead of unary minus, for sake of avoiding compiler warnings.
return operator+=(D(0)-value);
}
value_type operator++() {
return fetch_and_add(1)+1;
}
value_type operator--() {
return fetch_and_add(__TBB_MINUS_ONE(D))-1;
}
value_type operator++(int) {
return fetch_and_add(1);
}
value_type operator--(int) {
return fetch_and_add(__TBB_MINUS_ONE(D));
}
};
} /* Internal */
//! @endcond
//! Primary template for atomic.
/** See the Reference for details.
@ingroup synchronization */
template<typename T>
struct atomic: internal::atomic_impl<T> {
#if __TBB_ATOMIC_CTORS
atomic() = default;
constexpr atomic(T arg): internal::atomic_impl<T>(arg) {}
#endif
T operator=( T rhs ) {
// "this" required here in strict ISO C++ because store_with_release is a dependent name
return this->store_with_release(rhs);
}
atomic<T>& operator=( const atomic<T>& rhs ) {this->store_with_release(rhs); return *this;}
};
#if __TBB_ATOMIC_CTORS
#define __TBB_DECL_ATOMIC(T) \
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
atomic() = default; \
constexpr atomic(T arg): internal::atomic_impl_with_arithmetic<T,T,char>(arg) {} \
\
T operator=( T rhs ) {return store_with_release(rhs);} \
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
};
#else
#define __TBB_DECL_ATOMIC(T) \
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
T operator=( T rhs ) {return store_with_release(rhs);} \
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
};
#endif
#if __TBB_64BIT_ATOMICS
//TODO: consider adding non-default (and atomic) copy constructor for 32bit platform
__TBB_DECL_ATOMIC(__TBB_LONG_LONG)
__TBB_DECL_ATOMIC(unsigned __TBB_LONG_LONG)
#else
// test_atomic will verify that sizeof(long long)==8
#endif
__TBB_DECL_ATOMIC(long)
__TBB_DECL_ATOMIC(unsigned long)
#if _MSC_VER && !_WIN64
#if __TBB_ATOMIC_CTORS
/* Special version of __TBB_DECL_ATOMIC that avoids gratuitous warnings from cl /Wp64 option.
It is identical to __TBB_DECL_ATOMIC(unsigned) except that it replaces operator=(T)
with an operator=(U) that explicitly converts the U to a T. Types T and U should be
type synonyms on the platform. Type U should be the wider variant of T from the
perspective of /Wp64. */
#define __TBB_DECL_ATOMIC_ALT(T,U) \
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
atomic() = default ; \
constexpr atomic(T arg): internal::atomic_impl_with_arithmetic<T,T,char>(arg) {} \
T operator=( U rhs ) {return store_with_release(T(rhs));} \
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
};
#else
#define __TBB_DECL_ATOMIC_ALT(T,U) \
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
T operator=( U rhs ) {return store_with_release(T(rhs));} \
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
};
#endif
__TBB_DECL_ATOMIC_ALT(unsigned,size_t)
__TBB_DECL_ATOMIC_ALT(int,ptrdiff_t)
#else
__TBB_DECL_ATOMIC(unsigned)
__TBB_DECL_ATOMIC(int)
#endif /* _MSC_VER && !_WIN64 */
__TBB_DECL_ATOMIC(unsigned short)
__TBB_DECL_ATOMIC(short)
__TBB_DECL_ATOMIC(char)
__TBB_DECL_ATOMIC(signed char)
__TBB_DECL_ATOMIC(unsigned char)
#if !_MSC_VER || defined(_NATIVE_WCHAR_T_DEFINED)
__TBB_DECL_ATOMIC(wchar_t)
#endif /* _MSC_VER||!defined(_NATIVE_WCHAR_T_DEFINED) */
//! Specialization for atomic<T*> with arithmetic and operator->.
template<typename T> struct atomic<T*>: internal::atomic_impl_with_arithmetic<T*,ptrdiff_t,T> {
#if __TBB_ATOMIC_CTORS
atomic() = default ;
constexpr atomic(T* arg): internal::atomic_impl_with_arithmetic<T*,ptrdiff_t,T>(arg) {}
#endif
T* operator=( T* rhs ) {
// "this" required here in strict ISO C++ because store_with_release is a dependent name
return this->store_with_release(rhs);
}
atomic<T*>& operator=( const atomic<T*>& rhs ) {
this->store_with_release(rhs); return *this;
}
T* operator->() const {
return (*this);
}
};
//! Specialization for atomic<void*>, for sake of not allowing arithmetic or operator->.
template<> struct atomic<void*>: internal::atomic_impl<void*> {
#if __TBB_ATOMIC_CTORS
atomic() = default ;
constexpr atomic(void* arg): internal::atomic_impl<void*>(arg) {}
#endif
void* operator=( void* rhs ) {
// "this" required here in strict ISO C++ because store_with_release is a dependent name
return this->store_with_release(rhs);
}
atomic<void*>& operator=( const atomic<void*>& rhs ) {
this->store_with_release(rhs); return *this;
}
};
// Helpers to workaround ugly syntax of calling template member function of a
// template class with template argument dependent on template parameters.
template <memory_semantics M, typename T>
T load ( const atomic<T>& a ) { return a.template load<M>(); }
template <memory_semantics M, typename T>
void store ( atomic<T>& a, T value ) { a.template store<M>(value); }
namespace interface6{
//! Make an atomic for use in an initialization (list), as an alternative to zero-initialization or normal assignment.
template<typename T>
atomic<T> make_atomic(T t) {
atomic<T> a;
store<relaxed>(a,t);
return a;
}
}
using interface6::make_atomic;
namespace internal {
template<memory_semantics M, typename T >
void swap(atomic<T> & lhs, atomic<T> & rhs){
T tmp = load<M>(lhs);
store<M>(lhs,load<M>(rhs));
store<M>(rhs,tmp);
}
// only to aid in the gradual conversion of ordinary variables to proper atomics
template<typename T>
inline atomic<T>& as_atomic( T& t ) {
return (atomic<T>&)t;
}
} // namespace tbb::internal
} // namespace tbb
#if _MSC_VER && !__INTEL_COMPILER
#pragma warning (pop)
#endif // warnings 4244, 4267 are back
#endif /* __TBB_atomic_H */

View File

@@ -0,0 +1,159 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_blocked_range_H
#define __TBB_blocked_range_H
#include "tbb_stddef.h"
namespace tbb {
/** \page range_req Requirements on range concept
Class \c R implementing the concept of range must define:
- \code R::R( const R& ); \endcode Copy constructor
- \code R::~R(); \endcode Destructor
- \code bool R::is_divisible() const; \endcode True if range can be partitioned into two subranges
- \code bool R::empty() const; \endcode True if range is empty
- \code R::R( R& r, split ); \endcode Split range \c r into two subranges.
**/
//! A range over which to iterate.
/** @ingroup algorithms */
template<typename Value>
class blocked_range {
public:
//! Type of a value
/** Called a const_iterator for sake of algorithms that need to treat a blocked_range
as an STL container. */
typedef Value const_iterator;
//! Type for size of a range
typedef std::size_t size_type;
//! Construct range with default-constructed values for begin and end.
/** Requires that Value have a default constructor. */
blocked_range() : my_end(), my_begin() {}
//! Construct range over half-open interval [begin,end), with the given grainsize.
blocked_range( Value begin_, Value end_, size_type grainsize_=1 ) :
my_end(end_), my_begin(begin_), my_grainsize(grainsize_)
{
__TBB_ASSERT( my_grainsize>0, "grainsize must be positive" );
}
//! Beginning of range.
const_iterator begin() const {return my_begin;}
//! One past last value in range.
const_iterator end() const {return my_end;}
//! Size of the range
/** Unspecified if end()<begin(). */
size_type size() const {
__TBB_ASSERT( !(end()<begin()), "size() unspecified if end()<begin()" );
return size_type(my_end-my_begin);
}
//! The grain size for this range.
size_type grainsize() const {return my_grainsize;}
//------------------------------------------------------------------------
// Methods that implement Range concept
//------------------------------------------------------------------------
//! True if range is empty.
bool empty() const {return !(my_begin<my_end);}
//! True if range is divisible.
/** Unspecified if end()<begin(). */
bool is_divisible() const {return my_grainsize<size();}
//! Split range.
/** The new Range *this has the second part, the old range r has the first part.
Unspecified if end()<begin() or !is_divisible(). */
blocked_range( blocked_range& r, split ) :
my_end(r.my_end),
my_begin(do_split(r, split())),
my_grainsize(r.my_grainsize)
{
// only comparison 'less than' is required from values of blocked_range objects
__TBB_ASSERT( !(my_begin < r.my_end) && !(r.my_end < my_begin), "blocked_range has been split incorrectly" );
}
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
//! Static field to support proportional split
static const bool is_divisible_in_proportion = true;
//! Split range.
/** The new Range *this has the second part split according to specified proportion, the old range r has the first part.
Unspecified if end()<begin() or !is_divisible(). */
blocked_range( blocked_range& r, proportional_split& proportion ) :
my_end(r.my_end),
my_begin(do_split(r, proportion)),
my_grainsize(r.my_grainsize)
{
// only comparison 'less than' is required from values of blocked_range objects
__TBB_ASSERT( !(my_begin < r.my_end) && !(r.my_end < my_begin), "blocked_range has been split incorrectly" );
}
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
private:
/** NOTE: my_end MUST be declared before my_begin, otherwise the forking constructor will break. */
Value my_end;
Value my_begin;
size_type my_grainsize;
//! Auxiliary function used by forking constructor.
/** Using this function lets us not require that Value support assignment or default construction. */
static Value do_split( blocked_range& r, split )
{
__TBB_ASSERT( r.is_divisible(), "cannot split blocked_range that is not divisible" );
Value middle = r.my_begin + (r.my_end - r.my_begin) / 2u;
r.my_end = middle;
return middle;
}
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
static Value do_split( blocked_range& r, proportional_split& proportion )
{
__TBB_ASSERT( r.is_divisible(), "cannot split blocked_range that is not divisible" );
// usage of 32-bit floating point arithmetic is not enough to handle ranges of
// more than 2^24 iterations accurately. However, even on ranges with 2^64
// iterations the computational error approximately equals to 0.000001% which
// makes small impact on uniform distribution of such range's iterations (assuming
// all iterations take equal time to complete). See 'test_partitioner_whitebox'
// for implementation of an exact split algorithm
size_type right_part = size_type(float(r.size()) * float(proportion.right())
/ float(proportion.left() + proportion.right()) + 0.5f);
return r.my_end = Value(r.my_end - right_part);
}
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
template<typename RowValue, typename ColValue>
friend class blocked_range2d;
template<typename RowValue, typename ColValue, typename PageValue>
friend class blocked_range3d;
};
} // namespace tbb
#endif /* __TBB_blocked_range_H */

View File

@@ -0,0 +1,108 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_blocked_range2d_H
#define __TBB_blocked_range2d_H
#include "tbb_stddef.h"
#include "blocked_range.h"
namespace tbb {
//! A 2-dimensional range that models the Range concept.
/** @ingroup algorithms */
template<typename RowValue, typename ColValue=RowValue>
class blocked_range2d {
public:
//! Type for size of an iteration range
typedef blocked_range<RowValue> row_range_type;
typedef blocked_range<ColValue> col_range_type;
private:
row_range_type my_rows;
col_range_type my_cols;
public:
blocked_range2d( RowValue row_begin, RowValue row_end, typename row_range_type::size_type row_grainsize,
ColValue col_begin, ColValue col_end, typename col_range_type::size_type col_grainsize ) :
my_rows(row_begin,row_end,row_grainsize),
my_cols(col_begin,col_end,col_grainsize)
{
}
blocked_range2d( RowValue row_begin, RowValue row_end,
ColValue col_begin, ColValue col_end ) :
my_rows(row_begin,row_end),
my_cols(col_begin,col_end)
{
}
//! True if range is empty
bool empty() const {
// Yes, it is a logical OR here, not AND.
return my_rows.empty() || my_cols.empty();
}
//! True if range is divisible into two pieces.
bool is_divisible() const {
return my_rows.is_divisible() || my_cols.is_divisible();
}
blocked_range2d( blocked_range2d& r, split ) :
my_rows(r.my_rows),
my_cols(r.my_cols)
{
split split_obj;
do_split(r, split_obj);
}
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
//! Static field to support proportional split
static const bool is_divisible_in_proportion = true;
blocked_range2d( blocked_range2d& r, proportional_split& proportion ) :
my_rows(r.my_rows),
my_cols(r.my_cols)
{
do_split(r, proportion);
}
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
template <typename Split>
void do_split( blocked_range2d& r, Split& split_obj )
{
if( my_rows.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_rows.grainsize()) ) {
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
} else {
my_rows.my_begin = row_range_type::do_split(r.my_rows, split_obj);
}
}
//! The rows of the iteration space
const row_range_type& rows() const {return my_rows;}
//! The columns of the iteration space
const col_range_type& cols() const {return my_cols;}
};
} // namespace tbb
#endif /* __TBB_blocked_range2d_H */

View File

@@ -0,0 +1,128 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_blocked_range3d_H
#define __TBB_blocked_range3d_H
#include "tbb_stddef.h"
#include "blocked_range.h"
namespace tbb {
//! A 3-dimensional range that models the Range concept.
/** @ingroup algorithms */
template<typename PageValue, typename RowValue=PageValue, typename ColValue=RowValue>
class blocked_range3d {
public:
//! Type for size of an iteration range
typedef blocked_range<PageValue> page_range_type;
typedef blocked_range<RowValue> row_range_type;
typedef blocked_range<ColValue> col_range_type;
private:
page_range_type my_pages;
row_range_type my_rows;
col_range_type my_cols;
public:
blocked_range3d( PageValue page_begin, PageValue page_end,
RowValue row_begin, RowValue row_end,
ColValue col_begin, ColValue col_end ) :
my_pages(page_begin,page_end),
my_rows(row_begin,row_end),
my_cols(col_begin,col_end)
{
}
blocked_range3d( PageValue page_begin, PageValue page_end, typename page_range_type::size_type page_grainsize,
RowValue row_begin, RowValue row_end, typename row_range_type::size_type row_grainsize,
ColValue col_begin, ColValue col_end, typename col_range_type::size_type col_grainsize ) :
my_pages(page_begin,page_end,page_grainsize),
my_rows(row_begin,row_end,row_grainsize),
my_cols(col_begin,col_end,col_grainsize)
{
}
//! True if range is empty
bool empty() const {
// Yes, it is a logical OR here, not AND.
return my_pages.empty() || my_rows.empty() || my_cols.empty();
}
//! True if range is divisible into two pieces.
bool is_divisible() const {
return my_pages.is_divisible() || my_rows.is_divisible() || my_cols.is_divisible();
}
blocked_range3d( blocked_range3d& r, split ) :
my_pages(r.my_pages),
my_rows(r.my_rows),
my_cols(r.my_cols)
{
split split_obj;
do_split(r, split_obj);
}
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
//! Static field to support proportional split
static const bool is_divisible_in_proportion = true;
blocked_range3d( blocked_range3d& r, proportional_split& proportion ) :
my_pages(r.my_pages),
my_rows(r.my_rows),
my_cols(r.my_cols)
{
do_split(r, proportion);
}
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
template <typename Split>
void do_split( blocked_range3d& r, Split& split_obj)
{
if ( my_pages.size()*double(my_rows.grainsize()) < my_rows.size()*double(my_pages.grainsize()) ) {
if ( my_rows.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_rows.grainsize()) ) {
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
} else {
my_rows.my_begin = row_range_type::do_split(r.my_rows, split_obj);
}
} else {
if ( my_pages.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_pages.grainsize()) ) {
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
} else {
my_pages.my_begin = page_range_type::do_split(r.my_pages, split_obj);
}
}
}
//! The pages of the iteration space
const page_range_type& pages() const {return my_pages;}
//! The rows of the iteration space
const row_range_type& rows() const {return my_rows;}
//! The columns of the iteration space
const col_range_type& cols() const {return my_cols;}
};
} // namespace tbb
#endif /* __TBB_blocked_range3d_H */

View File

@@ -0,0 +1,256 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
#include "tbb/cache_aligned_allocator.h"
#include "tbb/tbb_allocator.h"
#include "tbb/tbb_exception.h"
#include "tbb_misc.h"
#include "dynamic_link.h"
#include <cstdlib>
#if _WIN32||_WIN64
#include "tbb/machine/windows_api.h"
#else
#include <dlfcn.h>
#endif /* _WIN32||_WIN64 */
using namespace std;
#if __TBB_WEAK_SYMBOLS_PRESENT
#pragma weak scalable_malloc
#pragma weak scalable_free
#pragma weak scalable_aligned_malloc
#pragma weak scalable_aligned_free
extern "C" {
void* scalable_malloc( size_t );
void scalable_free( void* );
void* scalable_aligned_malloc( size_t, size_t );
void scalable_aligned_free( void* );
}
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
namespace tbb {
namespace internal {
//! Dummy routine used for first indirect call via MallocHandler.
static void* DummyMalloc( size_t size );
//! Dummy routine used for first indirect call via FreeHandler.
static void DummyFree( void * ptr );
//! Handler for memory allocation
static void* (*MallocHandler)( size_t size ) = &DummyMalloc;
//! Handler for memory deallocation
static void (*FreeHandler)( void* pointer ) = &DummyFree;
//! Dummy routine used for first indirect call via padded_allocate_handler.
static void* dummy_padded_allocate( size_t bytes, size_t alignment );
//! Dummy routine used for first indirect call via padded_free_handler.
static void dummy_padded_free( void * ptr );
// ! Allocates memory using standard malloc. It is used when scalable_allocator is not available
static void* padded_allocate( size_t bytes, size_t alignment );
// ! Allocates memory using standard free. It is used when scalable_allocator is not available
static void padded_free( void* p );
//! Handler for padded memory allocation
static void* (*padded_allocate_handler)( size_t bytes, size_t alignment ) = &dummy_padded_allocate;
//! Handler for padded memory deallocation
static void (*padded_free_handler)( void* p ) = &dummy_padded_free;
//! Table describing how to link the handlers.
static const dynamic_link_descriptor MallocLinkTable[] = {
DLD(scalable_malloc, MallocHandler),
DLD(scalable_free, FreeHandler),
DLD(scalable_aligned_malloc, padded_allocate_handler),
DLD(scalable_aligned_free, padded_free_handler),
};
#if TBB_USE_DEBUG
#define DEBUG_SUFFIX "_debug"
#else
#define DEBUG_SUFFIX
#endif /* TBB_USE_DEBUG */
// MALLOCLIB_NAME is the name of the TBB memory allocator library.
#if _WIN32||_WIN64
#define MALLOCLIB_NAME "tbbmalloc" DEBUG_SUFFIX ".dll"
#elif __APPLE__
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX ".dylib"
#elif __FreeBSD__ || __NetBSD__ || __sun || _AIX || __ANDROID__
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX ".so"
#elif __linux__ // Note that order of these #elif's is important!
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX __TBB_STRING(.so.TBB_COMPATIBLE_INTERFACE_VERSION)
#else
#error Unknown OS
#endif
//! Initialize the allocation/free handler pointers.
/** Caller is responsible for ensuring this routine is called exactly once.
The routine attempts to dynamically link with the TBB memory allocator.
If that allocator is not found, it links to malloc and free. */
void initialize_handler_pointers() {
__TBB_ASSERT( MallocHandler==&DummyMalloc, NULL );
bool success = dynamic_link( MALLOCLIB_NAME, MallocLinkTable, 4 );
if( !success ) {
// If unsuccessful, set the handlers to the default routines.
// This must be done now, and not before FillDynamicLinks runs, because if other
// threads call the handlers, we want them to go through the DoOneTimeInitializations logic,
// which forces them to wait.
FreeHandler = &free;
MallocHandler = &malloc;
padded_allocate_handler = &padded_allocate;
padded_free_handler = &padded_free;
}
#if !__TBB_RML_STATIC
PrintExtraVersionInfo( "ALLOCATOR", success?"scalable_malloc":"malloc" );
#endif
}
static tbb::atomic<do_once_state> initialization_state;
void initialize_cache_aligned_allocator() {
atomic_do_once( &initialize_handler_pointers, initialization_state );
}
//! Executed on very first call through MallocHandler
static void* DummyMalloc( size_t size ) {
initialize_cache_aligned_allocator();
__TBB_ASSERT( MallocHandler!=&DummyMalloc, NULL );
return (*MallocHandler)( size );
}
//! Executed on very first call through FreeHandler
static void DummyFree( void * ptr ) {
initialize_cache_aligned_allocator();
__TBB_ASSERT( FreeHandler!=&DummyFree, NULL );
(*FreeHandler)( ptr );
}
//! Executed on very first call through padded_allocate_handler
static void* dummy_padded_allocate( size_t bytes, size_t alignment ) {
initialize_cache_aligned_allocator();
__TBB_ASSERT( padded_allocate_handler!=&dummy_padded_allocate, NULL );
return (*padded_allocate_handler)(bytes, alignment);
}
//! Executed on very first call through padded_free_handler
static void dummy_padded_free( void * ptr ) {
initialize_cache_aligned_allocator();
__TBB_ASSERT( padded_free_handler!=&dummy_padded_free, NULL );
(*padded_free_handler)( ptr );
}
static size_t NFS_LineSize = 128;
size_t NFS_GetLineSize() {
return NFS_LineSize;
}
#if _MSC_VER && !defined(__INTEL_COMPILER)
// unary minus operator applied to unsigned type, result still unsigned
#pragma warning( disable: 4146 4706 )
#endif
void* NFS_Allocate( size_t n, size_t element_size, void* /*hint*/ ) {
size_t m = NFS_LineSize;
__TBB_ASSERT( m<=NFS_MaxLineSize, "illegal value for NFS_LineSize" );
__TBB_ASSERT( (m & (m-1))==0, "must be power of two" );
size_t bytes = n*element_size;
if (bytes<n || bytes+m<bytes) {
// Overflow
throw_exception(eid_bad_alloc);
}
// scalable_aligned_malloc considers zero size request an error, and returns NULL
if (bytes==0) bytes = 1;
void* result = (*padded_allocate_handler)( bytes, m );
if (!result)
throw_exception(eid_bad_alloc);
__TBB_ASSERT( ((size_t)result&(m-1)) == 0, "The address returned isn't aligned to cache line size" );
return result;
}
void NFS_Free( void* p ) {
(*padded_free_handler)( p );
}
static void* padded_allocate( size_t bytes, size_t alignment ) {
unsigned char* result = NULL;
unsigned char* base = (unsigned char*)malloc(alignment+bytes);
if( base ) {
// Round up to the next line
result = (unsigned char*)((uintptr_t)(base+alignment)&-alignment);
// Record where block actually starts.
((uintptr_t*)result)[-1] = uintptr_t(base);
}
return result;
}
static void padded_free( void* p ) {
if( p ) {
__TBB_ASSERT( (uintptr_t)p>=0x4096, "attempt to free block not obtained from cache_aligned_allocator" );
// Recover where block actually starts
unsigned char* base = ((unsigned char**)p)[-1];
__TBB_ASSERT( (void*)((uintptr_t)(base+NFS_LineSize)&-NFS_LineSize)==p, "not allocated by NFS_Allocate?" );
free(base);
}
}
void* __TBB_EXPORTED_FUNC allocate_via_handler_v3( size_t n ) {
void* result = (*MallocHandler) (n);
if (!result) {
throw_exception(eid_bad_alloc);
}
return result;
}
void __TBB_EXPORTED_FUNC deallocate_via_handler_v3( void *p ) {
if( p ) {
(*FreeHandler)( p );
}
}
bool __TBB_EXPORTED_FUNC is_malloc_used_v3() {
if (MallocHandler == &DummyMalloc) {
void* void_ptr = (*MallocHandler)(1);
(*FreeHandler)(void_ptr);
}
__TBB_ASSERT( MallocHandler!=&DummyMalloc && FreeHandler!=&DummyFree, NULL );
// Cast to void avoids type mismatch errors on some compilers (e.g. __IBMCPP__)
__TBB_ASSERT( !(((void*)MallocHandler==(void*)&malloc) ^ ((void*)FreeHandler==(void*)&free)),
"Both shim pointers must refer to routines from the same package (either TBB or CRT)" );
return (void*)MallocHandler == (void*)&malloc;
}
} // namespace internal
} // namespace tbb

View File

@@ -0,0 +1,137 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_cache_aligned_allocator_H
#define __TBB_cache_aligned_allocator_H
#include <new>
#include "tbb_stddef.h"
#if __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
#include <utility> // std::forward
#endif
namespace tbb {
//! @cond INTERNAL
namespace internal {
//! Cache/sector line size.
/** @ingroup memory_allocation */
size_t __TBB_EXPORTED_FUNC NFS_GetLineSize();
//! Allocate memory on cache/sector line boundary.
/** @ingroup memory_allocation */
void* __TBB_EXPORTED_FUNC NFS_Allocate( size_t n_element, size_t element_size, void* hint );
//! Free memory allocated by NFS_Allocate.
/** Freeing a NULL pointer is allowed, but has no effect.
@ingroup memory_allocation */
void __TBB_EXPORTED_FUNC NFS_Free( void* );
}
//! @endcond
#if _MSC_VER && !defined(__INTEL_COMPILER)
// Workaround for erroneous "unreferenced parameter" warning in method destroy.
#pragma warning (push)
#pragma warning (disable: 4100)
#endif
//! Meets "allocator" requirements of ISO C++ Standard, Section 20.1.5
/** The members are ordered the same way they are in section 20.4.1
of the ISO C++ standard.
@ingroup memory_allocation */
template<typename T>
class cache_aligned_allocator {
public:
typedef typename internal::allocator_type<T>::value_type value_type;
typedef value_type* pointer;
typedef const value_type* const_pointer;
typedef value_type& reference;
typedef const value_type& const_reference;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
template<typename U> struct rebind {
typedef cache_aligned_allocator<U> other;
};
cache_aligned_allocator() throw() {}
cache_aligned_allocator( const cache_aligned_allocator& ) throw() {}
template<typename U> cache_aligned_allocator(const cache_aligned_allocator<U>&) throw() {}
pointer address(reference x) const {return &x;}
const_pointer address(const_reference x) const {return &x;}
//! Allocate space for n objects, starting on a cache/sector line.
pointer allocate( size_type n, const void* hint=0 ) {
// The "hint" argument is always ignored in NFS_Allocate thus const_cast shouldn't hurt
return pointer(internal::NFS_Allocate( n, sizeof(value_type), const_cast<void*>(hint) ));
}
//! Free block of memory that starts on a cache line
void deallocate( pointer p, size_type ) {
internal::NFS_Free(p);
}
//! Largest value for which method allocate might succeed.
size_type max_size() const throw() {
return (~size_t(0)-internal::NFS_MaxLineSize)/sizeof(value_type);
}
//! Copy-construct value at location pointed to by p.
#if __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
template<typename U, typename... Args>
void construct(U *p, Args&&... args)
{ ::new((void *)p) U(std::forward<Args>(args)...); }
#else // __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
#if __TBB_CPP11_RVALUE_REF_PRESENT
void construct( pointer p, value_type&& value ) {::new((void*)(p)) value_type(std::move(value));}
#endif
void construct( pointer p, const value_type& value ) {::new((void*)(p)) value_type(value);}
#endif // __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
//! Destroy value at location pointed to by p.
void destroy( pointer p ) {p->~value_type();}
};
#if _MSC_VER && !defined(__INTEL_COMPILER)
#pragma warning (pop)
#endif // warning 4100 is back
//! Analogous to std::allocator<void>, as defined in ISO C++ Standard, Section 20.4.1
/** @ingroup memory_allocation */
template<>
class cache_aligned_allocator<void> {
public:
typedef void* pointer;
typedef const void* const_pointer;
typedef void value_type;
template<typename U> struct rebind {
typedef cache_aligned_allocator<U> other;
};
};
template<typename T, typename U>
inline bool operator==( const cache_aligned_allocator<T>&, const cache_aligned_allocator<U>& ) {return true;}
template<typename T, typename U>
inline bool operator!=( const cache_aligned_allocator<T>&, const cache_aligned_allocator<U>& ) {return false;}
} // namespace tbb
#endif /* __TBB_cache_aligned_allocator_H */

View File

@@ -0,0 +1,115 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
/* The API to enable interoperability between Intel(R) Cilk(TM) Plus and
Intel(R) Threading Building Blocks. */
#ifndef CILK_TBB_INTEROP_H
#define CILK_TBB_INTEROP_H
#ifndef _WIN32
#ifdef IN_CILK_RUNTIME
#define CILK_EXPORT __attribute__((visibility("protected")))
#else
#define CILK_EXPORT /* nothing */
#endif
#else
#ifdef IN_CILK_RUNTIME
#define CILK_EXPORT __declspec(dllexport)
#else
#define CILK_EXPORT __declspec(dllimport)
#endif // IN_CILK_RUNTIME
#endif // _WIN32
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/* A return code. 0 indicates success */
typedef int __cilk_tbb_retcode;
enum __cilk_tbb_stack_op {
CILK_TBB_STACK_ORPHAN, // disconnecting stack from a thread
CILK_TBB_STACK_ADOPT, // reconnecting orphaned stack to a trhead
CILK_TBB_STACK_RELEASE // releasing stack
};
typedef __cilk_tbb_retcode (*__cilk_tbb_pfn_stack_op)(enum __cilk_tbb_stack_op, void* data);
typedef __cilk_tbb_retcode (*__cilk_tbb_pfn_unwatch_stacks)(void *data);
/* Each thunk structure has two pointers: "routine" and "data".
The caller of the thunk invokes *routine, passing "data" as the void* parameter. */
/* Thunk invoked by Intel Cilk Plus runtime (cilkrts) when it changes the relationship
between a stack and a thread. It does not matter what stack the thunk runs on.
The thread (not fiber) on which the thunk runs is important.
CILK_TBB_STACK_ORPHAN
The thunk must be invoked on the thread disconnecting itself from the stack.
Must "happen before" the stack is adopted elsewhere.
CILK_TBB_STACK_ADOPT
The thunk must be invoked on the thread adopting the stack.
CILK_TBB_STACK_RELEASE
The thunk must be invoked on the thread doing the releasing,
Must "happen before" the stack is used elsewhere.
When a non-empty stack is transfered between threads, the first thread must orphan it
and the second thread must adopt it.
An empty stack can be transfered similarly, or simply released by the first thread.
Here is a summary of the actions as transitions on a state machine.
watch ORPHAN
-->--> -->--
/ \ / \
(freed empty stack) (TBB sees stack running on thread) (stack in limbo)
| \ / \ / |
| --<-- --<-- |
^ RELEASE or ADOPT V
\ unwatch /
\ /
--------------------------<---------------------------
RELEASE
*/
struct __cilk_tbb_stack_op_thunk {
__cilk_tbb_pfn_stack_op routine;
void* data; /* Set by TBB */
};
/* Thunk invoked by TBB when it is no longer interested in watching the stack bound to the current thread. */
struct __cilk_tbb_unwatch_thunk {
__cilk_tbb_pfn_unwatch_stacks routine;
void* data;
};
/* Defined by cilkrts, called by TBB.
Requests that cilkrts invoke __cilk_tbb_stack_op_thunk when it orphans a stack.
cilkrts sets *u to a thunk that TBB should call when it is no longer interested in watching the stack. */
CILK_EXPORT
__cilk_tbb_retcode __cilkrts_watch_stack(struct __cilk_tbb_unwatch_thunk* u,
struct __cilk_tbb_stack_op_thunk o);
#ifdef __cplusplus
}
#endif /* __cplusplus */
#endif // CILK_TBB_INTEROP_H

View File

@@ -0,0 +1,72 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_combinable_H
#define __TBB_combinable_H
#include "enumerable_thread_specific.h"
#include "cache_aligned_allocator.h"
namespace tbb {
/** \name combinable
**/
//@{
//! Thread-local storage with optional reduction
/** @ingroup containers */
template <typename T>
class combinable {
private:
typedef typename tbb::cache_aligned_allocator<T> my_alloc;
typedef typename tbb::enumerable_thread_specific<T, my_alloc, ets_no_key> my_ets_type;
my_ets_type my_ets;
public:
combinable() { }
template <typename finit>
combinable( finit _finit) : my_ets(_finit) { }
//! destructor
~combinable() {
}
combinable(const combinable& other) : my_ets(other.my_ets) { }
combinable & operator=( const combinable & other) { my_ets = other.my_ets; return *this; }
void clear() { my_ets.clear(); }
T& local() { return my_ets.local(); }
T& local(bool & exists) { return my_ets.local(exists); }
// combine_func_t has signature T(T,T) or T(const T&, const T&)
template <typename combine_func_t>
T combine(combine_func_t f_combine) { return my_ets.combine(f_combine); }
// combine_func_t has signature void(T) or void(const T&)
template <typename combine_func_t>
void combine_each(combine_func_t f_combine) { my_ets.combine_each(f_combine); }
};
} // namespace tbb
#endif /* __TBB_combinable_H */

View File

@@ -0,0 +1,457 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_condition_variable_H
#define __TBB_condition_variable_H
#if _WIN32||_WIN64
#include "../machine/windows_api.h"
namespace tbb {
namespace interface5 {
namespace internal {
struct condition_variable_using_event
{
//! Event for blocking waiting threads.
HANDLE event;
//! Protects invariants involving n_waiters, release_count, and epoch.
CRITICAL_SECTION mutex;
//! Number of threads waiting on this condition variable
int n_waiters;
//! Number of threads remaining that should no longer wait on this condition variable.
int release_count;
//! To keep threads from waking up prematurely with earlier signals.
unsigned epoch;
};
}}} // namespace tbb::interface5::internal
#ifndef CONDITION_VARIABLE_INIT
typedef void* CONDITION_VARIABLE;
typedef CONDITION_VARIABLE* PCONDITION_VARIABLE;
#endif
#else /* if not _WIN32||_WIN64 */
#include <errno.h> // some systems need it for ETIMEDOUT
#include <pthread.h>
#if __linux__
#include <ctime>
#else /* generic Unix */
#include <sys/time.h>
#endif
#endif /* _WIN32||_WIN64 */
#include "../tbb_stddef.h"
#include "../mutex.h"
#include "../tbb_thread.h"
#include "../tbb_exception.h"
#include "../tbb_profiling.h"
namespace tbb {
namespace interface5 {
// C++0x standard working draft 30.4.3
// Lock tag types
struct defer_lock_t { }; //! do not acquire ownership of the mutex
struct try_to_lock_t { }; //! try to acquire ownership of the mutex without blocking
struct adopt_lock_t { }; //! assume the calling thread has already
const defer_lock_t defer_lock = {};
const try_to_lock_t try_to_lock = {};
const adopt_lock_t adopt_lock = {};
// C++0x standard working draft 30.4.3.1
//! lock_guard
template<typename M>
class lock_guard : tbb::internal::no_copy {
public:
//! mutex type
typedef M mutex_type;
//! Constructor
/** precondition: If mutex_type is not a recursive mutex, the calling thread
does not own the mutex m. */
explicit lock_guard(mutex_type& m) : pm(m) {m.lock();}
//! Adopt_lock constructor
/** precondition: the calling thread owns the mutex m. */
lock_guard(mutex_type& m, adopt_lock_t) : pm(m) {}
//! Destructor
~lock_guard() { pm.unlock(); }
private:
mutex_type& pm;
};
// C++0x standard working draft 30.4.3.2
//! unique_lock
template<typename M>
class unique_lock : tbb::internal::no_copy {
friend class condition_variable;
public:
typedef M mutex_type;
// 30.4.3.2.1 construct/copy/destroy
// NB: Without constructors that take an r-value reference to a unique_lock, the following constructor is of little use.
//! Constructor
/** postcondition: pm==0 && owns==false */
unique_lock() : pm(NULL), owns(false) {}
//! Constructor
/** precondition: if mutex_type is not a recursive mutex, the calling thread
does not own the mutex m. If the precondition is not met, a deadlock occurs.
postcondition: pm==&m and owns==true */
explicit unique_lock(mutex_type& m) : pm(&m) {m.lock(); owns=true;}
//! Defer_lock constructor
/** postcondition: pm==&m and owns==false */
unique_lock(mutex_type& m, defer_lock_t) : pm(&m), owns(false) {}
//! Try_to_lock constructor
/** precondition: if mutex_type is not a recursive mutex, the calling thread
does not own the mutex m. If the precondition is not met, a deadlock occurs.
postcondition: pm==&m and owns==res where res is the value returned by
the call to m.try_lock(). */
unique_lock(mutex_type& m, try_to_lock_t) : pm(&m) {owns = m.try_lock();}
//! Adopt_lock constructor
/** precondition: the calling thread owns the mutex. If it does not, mutex->unlock() would fail.
postcondition: pm==&m and owns==true */
unique_lock(mutex_type& m, adopt_lock_t) : pm(&m), owns(true) {}
//! Timed unique_lock acquisition.
/** To avoid requiring support for namespace chrono, this method deviates from the working draft in that
it uses tbb::tick_count::interval_t to specify the time duration. */
unique_lock(mutex_type& m, const tick_count::interval_t &i) : pm(&m) {owns = try_lock_for( i );}
//! Destructor
~unique_lock() { if( owns ) pm->unlock(); }
// 30.4.3.2.2 locking
//! Lock the mutex and own it.
void lock() {
if( pm ) {
if( !owns ) {
pm->lock();
owns = true;
} else
throw_exception_v4( tbb::internal::eid_possible_deadlock );
} else
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
__TBB_ASSERT( owns, NULL );
}
//! Try to lock the mutex.
/** If successful, note that this lock owns it. Otherwise, set it false. */
bool try_lock() {
if( pm ) {
if( !owns )
owns = pm->try_lock();
else
throw_exception_v4( tbb::internal::eid_possible_deadlock );
} else
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
return owns;
}
//! Try to lock the mutex.
bool try_lock_for( const tick_count::interval_t &i );
//! Unlock the mutex
/** And note that this lock no longer owns it. */
void unlock() {
if( owns ) {
pm->unlock();
owns = false;
} else
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
__TBB_ASSERT( !owns, NULL );
}
// 30.4.3.2.3 modifiers
//! Swap the two unique locks
void swap(unique_lock& u) {
mutex_type* t_pm = u.pm; u.pm = pm; pm = t_pm;
bool t_owns = u.owns; u.owns = owns; owns = t_owns;
}
//! Release control over the mutex.
mutex_type* release() {
mutex_type* o_pm = pm;
pm = NULL;
owns = false;
return o_pm;
}
// 30.4.3.2.4 observers
//! Does this lock own the mutex?
bool owns_lock() const { return owns; }
// TODO: Un-comment 'explicit' when the last non-C++0x compiler support is dropped
//! Does this lock own the mutex?
/*explicit*/ operator bool() const { return owns; }
//! Return the mutex that this lock currently has.
mutex_type* mutex() const { return pm; }
private:
mutex_type* pm;
bool owns;
};
template<typename M>
bool unique_lock<M>::try_lock_for( const tick_count::interval_t &i)
{
const int unique_lock_tick = 100; /* microseconds; 0.1 milliseconds */
// the smallest wait-time is 0.1 milliseconds.
bool res = pm->try_lock();
int duration_in_micro;
if( !res && (duration_in_micro=int(i.seconds()*1e6))>unique_lock_tick ) {
tick_count::interval_t i_100( double(unique_lock_tick)/1e6 /* seconds */); // 100 microseconds = 0.1*10E-3
do {
this_tbb_thread::sleep(i_100); // sleep for 100 micro seconds
duration_in_micro -= unique_lock_tick;
res = pm->try_lock();
} while( !res && duration_in_micro>unique_lock_tick );
}
return (owns=res);
}
//! Swap the two unique locks that have the mutexes of same type
template<typename M>
void swap(unique_lock<M>& x, unique_lock<M>& y) { x.swap( y ); }
namespace internal {
#if _WIN32||_WIN64
union condvar_impl_t {
condition_variable_using_event cv_event;
CONDITION_VARIABLE cv_native;
};
void __TBB_EXPORTED_FUNC internal_initialize_condition_variable( condvar_impl_t& cv );
void __TBB_EXPORTED_FUNC internal_destroy_condition_variable( condvar_impl_t& cv );
void __TBB_EXPORTED_FUNC internal_condition_variable_notify_one( condvar_impl_t& cv );
void __TBB_EXPORTED_FUNC internal_condition_variable_notify_all( condvar_impl_t& cv );
bool __TBB_EXPORTED_FUNC internal_condition_variable_wait( condvar_impl_t& cv, mutex* mtx, const tick_count::interval_t* i = NULL );
#else /* if !(_WIN32||_WIN64), i.e., POSIX threads */
typedef pthread_cond_t condvar_impl_t;
#endif
} // namespace internal
//! cv_status
/** C++0x standard working draft 30.5 */
enum cv_status { no_timeout, timeout };
//! condition variable
/** C++0x standard working draft 30.5.1
@ingroup synchronization */
class condition_variable : tbb::internal::no_copy {
public:
//! Constructor
condition_variable() {
#if _WIN32||_WIN64
internal_initialize_condition_variable( my_cv );
#else
pthread_cond_init( &my_cv, NULL );
#endif
}
//! Destructor
~condition_variable() {
//precondition: There shall be no thread blocked on *this.
#if _WIN32||_WIN64
internal_destroy_condition_variable( my_cv );
#else
pthread_cond_destroy( &my_cv );
#endif
}
//! Notify one thread and wake it up
void notify_one() {
#if _WIN32||_WIN64
internal_condition_variable_notify_one( my_cv );
#else
pthread_cond_signal( &my_cv );
#endif
}
//! Notify all threads
void notify_all() {
#if _WIN32||_WIN64
internal_condition_variable_notify_all( my_cv );
#else
pthread_cond_broadcast( &my_cv );
#endif
}
//! Release the mutex associated with the lock and wait on this condition variable
void wait(unique_lock<mutex>& lock);
//! Wait on this condition variable while pred is false
template <class Predicate>
void wait(unique_lock<mutex>& lock, Predicate pred) {
while( !pred() )
wait( lock );
}
//! Timed version of wait()
cv_status wait_for(unique_lock<mutex>& lock, const tick_count::interval_t &i );
//! Timed version of the predicated wait
/** The loop terminates when pred() returns true or when the time duration specified by rel_time (i) has elapsed. */
template<typename Predicate>
bool wait_for(unique_lock<mutex>& lock, const tick_count::interval_t &i, Predicate pred)
{
while( !pred() ) {
cv_status st = wait_for( lock, i );
if( st==timeout )
return pred();
}
return true;
}
// C++0x standard working draft. 30.2.3
typedef internal::condvar_impl_t* native_handle_type;
native_handle_type native_handle() { return (native_handle_type) &my_cv; }
private:
internal::condvar_impl_t my_cv;
};
#if _WIN32||_WIN64
inline void condition_variable::wait( unique_lock<mutex>& lock )
{
__TBB_ASSERT( lock.owns, NULL );
lock.owns = false;
if( !internal_condition_variable_wait( my_cv, lock.mutex() ) ) {
int ec = GetLastError();
// on Windows 7, SleepConditionVariableCS() may return ERROR_TIMEOUT while the doc says it returns WAIT_TIMEOUT
__TBB_ASSERT_EX( ec!=WAIT_TIMEOUT&&ec!=ERROR_TIMEOUT, NULL );
lock.owns = true;
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
}
lock.owns = true;
}
inline cv_status condition_variable::wait_for( unique_lock<mutex>& lock, const tick_count::interval_t& i )
{
cv_status rc = no_timeout;
__TBB_ASSERT( lock.owns, NULL );
lock.owns = false;
// condvar_wait could be SleepConditionVariableCS (or SleepConditionVariableSRW) or our own pre-vista cond_var_wait()
if( !internal_condition_variable_wait( my_cv, lock.mutex(), &i ) ) {
int ec = GetLastError();
if( ec==WAIT_TIMEOUT || ec==ERROR_TIMEOUT )
rc = timeout;
else {
lock.owns = true;
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
}
}
lock.owns = true;
return rc;
}
#else /* !(_WIN32||_WIN64) */
inline void condition_variable::wait( unique_lock<mutex>& lock )
{
__TBB_ASSERT( lock.owns, NULL );
lock.owns = false;
if( pthread_cond_wait( &my_cv, lock.mutex()->native_handle() ) ) {
lock.owns = true;
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
}
// upon successful return, the mutex has been locked and is owned by the calling thread.
lock.owns = true;
}
inline cv_status condition_variable::wait_for( unique_lock<mutex>& lock, const tick_count::interval_t& i )
{
#if __linux__
struct timespec req;
double sec = i.seconds();
clock_gettime( CLOCK_REALTIME, &req );
req.tv_sec += static_cast<long>(sec);
req.tv_nsec += static_cast<long>( (sec - static_cast<long>(sec))*1e9 );
#else /* generic Unix */
struct timeval tv;
struct timespec req;
double sec = i.seconds();
int status = gettimeofday(&tv, NULL);
__TBB_ASSERT_EX( status==0, "gettimeofday failed" );
req.tv_sec = tv.tv_sec + static_cast<long>(sec);
req.tv_nsec = tv.tv_usec*1000 + static_cast<long>( (sec - static_cast<long>(sec))*1e9 );
#endif /*(choice of OS) */
if( req.tv_nsec>=1e9 ) {
req.tv_sec += 1;
req.tv_nsec -= static_cast<long int>(1e9);
}
__TBB_ASSERT( 0<=req.tv_nsec && req.tv_nsec<1e9, NULL );
int ec;
cv_status rc = no_timeout;
__TBB_ASSERT( lock.owns, NULL );
lock.owns = false;
if( ( ec=pthread_cond_timedwait( &my_cv, lock.mutex()->native_handle(), &req ) ) ) {
if( ec==ETIMEDOUT )
rc = timeout;
else {
__TBB_ASSERT( lock.try_lock()==false, NULL );
lock.owns = true;
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
}
}
lock.owns = true;
return rc;
}
#endif /* !(_WIN32||_WIN64) */
} // namespace interface5
__TBB_DEFINE_PROFILING_SET_NAME(interface5::condition_variable)
} // namespace tbb
#if TBB_IMPLEMENT_CPP0X
namespace std {
using tbb::interface5::defer_lock_t;
using tbb::interface5::try_to_lock_t;
using tbb::interface5::adopt_lock_t;
using tbb::interface5::defer_lock;
using tbb::interface5::try_to_lock;
using tbb::interface5::adopt_lock;
using tbb::interface5::lock_guard;
using tbb::interface5::unique_lock;
using tbb::interface5::swap; /* this is for void std::swap(unique_lock<M>&,unique_lock<M>&) */
using tbb::interface5::condition_variable;
using tbb::interface5::cv_status;
using tbb::interface5::timeout;
using tbb::interface5::no_timeout;
} // namespace std
#endif /* TBB_IMPLEMENT_CPP0X */
#endif /* __TBB_condition_variable_H */

View File

@@ -0,0 +1,62 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_compat_ppl_H
#define __TBB_compat_ppl_H
#include "../task_group.h"
#include "../parallel_invoke.h"
#include "../parallel_for_each.h"
#include "../parallel_for.h"
#include "../tbb_exception.h"
#include "../critical_section.h"
#include "../reader_writer_lock.h"
#include "../combinable.h"
namespace Concurrency {
#if __TBB_TASK_GROUP_CONTEXT
using tbb::task_handle;
using tbb::task_group_status;
using tbb::task_group;
using tbb::structured_task_group;
using tbb::invalid_multiple_scheduling;
using tbb::missing_wait;
using tbb::make_task;
using tbb::not_complete;
using tbb::complete;
using tbb::canceled;
using tbb::is_current_task_group_canceling;
#endif /* __TBB_TASK_GROUP_CONTEXT */
using tbb::parallel_invoke;
using tbb::strict_ppl::parallel_for;
using tbb::parallel_for_each;
using tbb::critical_section;
using tbb::reader_writer_lock;
using tbb::combinable;
using tbb::improper_lock;
} // namespace Concurrency
#endif /* __TBB_compat_ppl_H */

View File

@@ -0,0 +1,46 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_thread_H
#define __TBB_thread_H
#include "../tbb_thread.h"
#if TBB_IMPLEMENT_CPP0X
namespace std {
typedef tbb::tbb_thread thread;
namespace this_thread {
using tbb::this_tbb_thread::get_id;
using tbb::this_tbb_thread::yield;
inline void sleep_for(const tbb::tick_count::interval_t& rel_time) {
tbb::internal::thread_sleep_v3( rel_time );
}
}
}
#endif /* TBB_IMPLEMENT_CPP0X */
#endif /* __TBB_thread_H */

View File

@@ -0,0 +1,488 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_tuple_H
#define __TBB_tuple_H
#include <utility>
#include "../tbb_stddef.h"
// build preprocessor variables for varying number of arguments
// Need the leading comma so the empty __TBB_T_PACK will not cause a syntax error.
#if __TBB_VARIADIC_MAX <= 5
#define __TBB_T_PACK
#define __TBB_U_PACK
#define __TBB_TYPENAME_T_PACK
#define __TBB_TYPENAME_U_PACK
#define __TBB_NULL_TYPE_PACK
#define __TBB_REF_T_PARAM_PACK
#define __TBB_CONST_REF_T_PARAM_PACK
#define __TBB_T_PARAM_LIST_PACK
#define __TBB_CONST_NULL_REF_PACK
//
#elif __TBB_VARIADIC_MAX == 6
#define __TBB_T_PACK ,__T5
#define __TBB_U_PACK ,__U5
#define __TBB_TYPENAME_T_PACK , typename __T5
#define __TBB_TYPENAME_U_PACK , typename __U5
#define __TBB_NULL_TYPE_PACK , null_type
#define __TBB_REF_T_PARAM_PACK ,__T5& t5
#define __TBB_CONST_REF_T_PARAM_PACK ,const __T5& t5
#define __TBB_T_PARAM_LIST_PACK ,t5
#define __TBB_CONST_NULL_REF_PACK , const null_type&
//
#elif __TBB_VARIADIC_MAX == 7
#define __TBB_T_PACK ,__T5, __T6
#define __TBB_U_PACK ,__U5, __U6
#define __TBB_TYPENAME_T_PACK , typename __T5 , typename __T6
#define __TBB_TYPENAME_U_PACK , typename __U5 , typename __U6
#define __TBB_NULL_TYPE_PACK , null_type, null_type
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6
#define __TBB_CONST_REF_T_PARAM_PACK ,const __T5& t5, const __T6& t6
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&
//
#elif __TBB_VARIADIC_MAX == 8
#define __TBB_T_PACK ,__T5, __T6, __T7
#define __TBB_U_PACK ,__U5, __U6, __U7
#define __TBB_TYPENAME_T_PACK , typename __T5 , typename __T6, typename __T7
#define __TBB_TYPENAME_U_PACK , typename __U5 , typename __U6, typename __U7
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&
//
#elif __TBB_VARIADIC_MAX == 9
#define __TBB_T_PACK ,__T5, __T6, __T7, __T8
#define __TBB_U_PACK ,__U5, __U6, __U7, __U8
#define __TBB_TYPENAME_T_PACK , typename __T5, typename __T6, typename __T7, typename __T8
#define __TBB_TYPENAME_U_PACK , typename __U5, typename __U6, typename __U7, typename __U8
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type, null_type
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7, __T8& t8
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7, const __T8& t8
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7 ,t8
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&, const null_type&
//
#elif __TBB_VARIADIC_MAX >= 10
#define __TBB_T_PACK ,__T5, __T6, __T7, __T8, __T9
#define __TBB_U_PACK ,__U5, __U6, __U7, __U8, __U9
#define __TBB_TYPENAME_T_PACK , typename __T5, typename __T6, typename __T7, typename __T8, typename __T9
#define __TBB_TYPENAME_U_PACK , typename __U5, typename __U6, typename __U7, typename __U8, typename __U9
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type, null_type, null_type
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7, __T8& t8, __T9& t9
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7, const __T8& t8, const __T9& t9
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7 ,t8 ,t9
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&, const null_type&, const null_type&
#endif
namespace tbb {
namespace interface5 {
namespace internal {
struct null_type { };
}
using internal::null_type;
// tuple forward declaration
template <typename __T0=null_type, typename __T1=null_type, typename __T2=null_type,
typename __T3=null_type, typename __T4=null_type
#if __TBB_VARIADIC_MAX >= 6
, typename __T5=null_type
#if __TBB_VARIADIC_MAX >= 7
, typename __T6=null_type
#if __TBB_VARIADIC_MAX >= 8
, typename __T7=null_type
#if __TBB_VARIADIC_MAX >= 9
, typename __T8=null_type
#if __TBB_VARIADIC_MAX >= 10
, typename __T9=null_type
#endif
#endif
#endif
#endif
#endif
>
class tuple;
namespace internal {
// const null_type temp
inline const null_type cnull() { return null_type(); }
// cons forward declaration
template <typename __HT, typename __TT> struct cons;
// type of a component of the cons
template<int __N, typename __T>
struct component {
typedef typename __T::tail_type next;
typedef typename component<__N-1,next>::type type;
};
template<typename __T>
struct component<0,__T> {
typedef typename __T::head_type type;
};
template<>
struct component<0,null_type> {
typedef null_type type;
};
// const version of component
template<int __N, typename __T>
struct component<__N, const __T>
{
typedef typename __T::tail_type next;
typedef const typename component<__N-1,next>::type type;
};
template<typename __T>
struct component<0, const __T>
{
typedef const typename __T::head_type type;
};
// helper class for getting components of cons
template< int __N>
struct get_helper {
template<typename __HT, typename __TT>
inline static typename component<__N, cons<__HT,__TT> >::type& get(cons<__HT,__TT>& ti) {
return get_helper<__N-1>::get(ti.tail);
}
template<typename __HT, typename __TT>
inline static typename component<__N, cons<__HT,__TT> >::type const& get(const cons<__HT,__TT>& ti) {
return get_helper<__N-1>::get(ti.tail);
}
};
template<>
struct get_helper<0> {
template<typename __HT, typename __TT>
inline static typename component<0, cons<__HT,__TT> >::type& get(cons<__HT,__TT>& ti) {
return ti.head;
}
template<typename __HT, typename __TT>
inline static typename component<0, cons<__HT,__TT> >::type const& get(const cons<__HT,__TT>& ti) {
return ti.head;
}
};
// traits adaptor
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK>
struct tuple_traits {
typedef cons <__T0, typename tuple_traits<__T1, __T2, __T3, __T4 __TBB_T_PACK , null_type>::U > U;
};
template <typename __T0>
struct tuple_traits<__T0, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > {
typedef cons<__T0, null_type> U;
};
template<>
struct tuple_traits<null_type, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > {
typedef null_type U;
};
// core cons defs
template <typename __HT, typename __TT>
struct cons{
typedef __HT head_type;
typedef __TT tail_type;
head_type head;
tail_type tail;
static const int length = 1 + tail_type::length;
// default constructors
explicit cons() : head(), tail() { }
// non-default constructors
cons(head_type& h, const tail_type& t) : head(h), tail(t) { }
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
cons(const __T0& t0, const __T1& t1, const __T2& t2, const __T3& t3, const __T4& t4 __TBB_CONST_REF_T_PARAM_PACK) :
head(t0), tail(t1, t2, t3, t4 __TBB_T_PARAM_LIST_PACK, cnull()) { }
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
cons(__T0& t0, __T1& t1, __T2& t2, __T3& t3, __T4& t4 __TBB_REF_T_PARAM_PACK) :
head(t0), tail(t1, t2, t3, t4 __TBB_T_PARAM_LIST_PACK , cnull()) { }
template <typename __HT1, typename __TT1>
cons(const cons<__HT1,__TT1>& other) : head(other.head), tail(other.tail) { }
cons& operator=(const cons& other) { head = other.head; tail = other.tail; return *this; }
friend bool operator==(const cons& me, const cons& other) {
return me.head == other.head && me.tail == other.tail;
}
friend bool operator<(const cons& me, const cons& other) {
return me.head < other.head || (!(other.head < me.head) && me.tail < other.tail);
}
friend bool operator>(const cons& me, const cons& other) { return other<me; }
friend bool operator!=(const cons& me, const cons& other) { return !(me==other); }
friend bool operator>=(const cons& me, const cons& other) { return !(me<other); }
friend bool operator<=(const cons& me, const cons& other) { return !(me>other); }
template<typename __HT1, typename __TT1>
friend bool operator==(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) {
return me.head == other.head && me.tail == other.tail;
}
template<typename __HT1, typename __TT1>
friend bool operator<(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) {
return me.head < other.head || (!(other.head < me.head) && me.tail < other.tail);
}
template<typename __HT1, typename __TT1>
friend bool operator>(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return other<me; }
template<typename __HT1, typename __TT1>
friend bool operator!=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me==other); }
template<typename __HT1, typename __TT1>
friend bool operator>=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me<other); }
template<typename __HT1, typename __TT1>
friend bool operator<=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me>other); }
}; // cons
template <typename __HT>
struct cons<__HT,null_type> {
typedef __HT head_type;
typedef null_type tail_type;
head_type head;
static const int length = 1;
// default constructor
cons() : head() { /*std::cout << "default constructor 1\n";*/ }
cons(const null_type&, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head() { /*std::cout << "default constructor 2\n";*/ }
// non-default constructor
template<typename __T1>
cons(__T1& t1, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(t1) { /*std::cout << "non-default a1, t1== " << t1 << "\n";*/}
cons(head_type& h, const null_type& = null_type() ) : head(h) { }
cons(const head_type& t0, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(t0) { }
// converting constructor
template<typename __HT1>
cons(__HT1 h1, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(h1) { }
// copy constructor
template<typename __HT1>
cons( const cons<__HT1, null_type>& other) : head(other.head) { }
// assignment operator
cons& operator=(const cons& other) { head = other.head; return *this; }
friend bool operator==(const cons& me, const cons& other) { return me.head == other.head; }
friend bool operator<(const cons& me, const cons& other) { return me.head < other.head; }
friend bool operator>(const cons& me, const cons& other) { return other<me; }
friend bool operator!=(const cons& me, const cons& other) {return !(me==other); }
friend bool operator<=(const cons& me, const cons& other) {return !(me>other); }
friend bool operator>=(const cons& me, const cons& other) {return !(me<other); }
template<typename __HT1>
friend bool operator==(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) {
return me.head == other.head;
}
template<typename __HT1>
friend bool operator<(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) {
return me.head < other.head;
}
template<typename __HT1>
friend bool operator>(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return other<me; }
template<typename __HT1>
friend bool operator!=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me==other); }
template<typename __HT1>
friend bool operator<=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me>other); }
template<typename __HT1>
friend bool operator>=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me<other); }
}; // cons
template <>
struct cons<null_type,null_type> { typedef null_type tail_type; static const int length = 0; };
// wrapper for default constructor
template<typename __T>
inline const __T wrap_dcons(__T*) { return __T(); }
} // namespace internal
// tuple definition
template<typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
class tuple : public internal::tuple_traits<__T0, __T1, __T2, __T3, __T4 __TBB_T_PACK >::U {
// friends
template <typename __T> friend class tuple_size;
template<int __N, typename __T> friend struct tuple_element;
// stl components
typedef tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > value_type;
typedef value_type *pointer;
typedef const value_type *const_pointer;
typedef value_type &reference;
typedef const value_type &const_reference;
typedef size_t size_type;
typedef typename internal::tuple_traits<__T0,__T1,__T2,__T3, __T4 __TBB_T_PACK >::U my_cons;
public:
tuple(const __T0& t0=internal::wrap_dcons((__T0*)NULL)
,const __T1& t1=internal::wrap_dcons((__T1*)NULL)
,const __T2& t2=internal::wrap_dcons((__T2*)NULL)
,const __T3& t3=internal::wrap_dcons((__T3*)NULL)
,const __T4& t4=internal::wrap_dcons((__T4*)NULL)
#if __TBB_VARIADIC_MAX >= 6
,const __T5& t5=internal::wrap_dcons((__T5*)NULL)
#if __TBB_VARIADIC_MAX >= 7
,const __T6& t6=internal::wrap_dcons((__T6*)NULL)
#if __TBB_VARIADIC_MAX >= 8
,const __T7& t7=internal::wrap_dcons((__T7*)NULL)
#if __TBB_VARIADIC_MAX >= 9
,const __T8& t8=internal::wrap_dcons((__T8*)NULL)
#if __TBB_VARIADIC_MAX >= 10
,const __T9& t9=internal::wrap_dcons((__T9*)NULL)
#endif
#endif
#endif
#endif
#endif
) :
my_cons(t0,t1,t2,t3,t4 __TBB_T_PARAM_LIST_PACK) { }
template<int __N>
struct internal_tuple_element {
typedef typename internal::component<__N,my_cons>::type type;
};
template<int __N>
typename internal_tuple_element<__N>::type& get() { return internal::get_helper<__N>::get(*this); }
template<int __N>
typename internal_tuple_element<__N>::type const& get() const { return internal::get_helper<__N>::get(*this); }
template<typename __U1, typename __U2>
tuple& operator=(const internal::cons<__U1,__U2>& other) {
my_cons::operator=(other);
return *this;
}
template<typename __U1, typename __U2>
tuple& operator=(const std::pair<__U1,__U2>& other) {
// __TBB_ASSERT(tuple_size<value_type>::value == 2, "Invalid size for pair to tuple assignment");
this->head = other.first;
this->tail.head = other.second;
return *this;
}
friend bool operator==(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)==(other);}
friend bool operator<(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)<(other);}
friend bool operator>(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)>(other);}
friend bool operator!=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)!=(other);}
friend bool operator>=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)>=(other);}
friend bool operator<=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)<=(other);}
}; // tuple
// empty tuple
template<>
class tuple<null_type, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > : public null_type {
};
// helper classes
template < typename __T>
class tuple_size {
public:
static const size_t value = 1 + tuple_size<typename __T::tail_type>::value;
};
template <>
class tuple_size<tuple<> > {
public:
static const size_t value = 0;
};
template <>
class tuple_size<null_type> {
public:
static const size_t value = 0;
};
template<int __N, typename __T>
struct tuple_element {
typedef typename internal::component<__N, typename __T::my_cons>::type type;
};
template<int __N, typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
inline static typename tuple_element<__N,tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > >::type&
get(tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK >& t) { return internal::get_helper<__N>::get(t); }
template<int __N, typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
inline static typename tuple_element<__N,tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > >::type const&
get(const tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK >& t) { return internal::get_helper<__N>::get(t); }
} // interface5
} // tbb
#if !__TBB_CPP11_TUPLE_PRESENT
namespace tbb {
namespace flow {
using tbb::interface5::tuple;
using tbb::interface5::tuple_size;
using tbb::interface5::tuple_element;
using tbb::interface5::get;
}
}
#endif
#undef __TBB_T_PACK
#undef __TBB_U_PACK
#undef __TBB_TYPENAME_T_PACK
#undef __TBB_TYPENAME_U_PACK
#undef __TBB_NULL_TYPE_PACK
#undef __TBB_REF_T_PARAM_PACK
#undef __TBB_CONST_REF_T_PARAM_PACK
#undef __TBB_T_PARAM_LIST_PACK
#undef __TBB_CONST_NULL_REF_PACK
#endif /* __TBB_tuple_H */

View File

@@ -0,0 +1,58 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/concurrent_hash_map.h"
namespace tbb {
namespace internal {
#if !TBB_NO_LEGACY
struct hash_map_segment_base {
typedef spin_rw_mutex segment_mutex_t;
//! Type of a hash code.
typedef size_t hashcode_t;
//! Log2 of n_segment
static const size_t n_segment_bits = 6;
//! Maximum size of array of chains
static const size_t max_physical_size = size_t(1)<<(8*sizeof(hashcode_t)-n_segment_bits);
//! Mutex that protects this segment
segment_mutex_t my_mutex;
// Number of nodes
atomic<size_t> my_logical_size;
// Size of chains
/** Always zero or a power of two */
size_t my_physical_size;
//! True if my_logical_size>=my_physical_size.
/** Used to support Intel(R) Thread Checker. */
bool __TBB_EXPORTED_METHOD internal_grow_predicate() const;
};
bool hash_map_segment_base::internal_grow_predicate() const {
// Intel(R) Thread Checker considers the following reads to be races, so we hide them in the
// library so that Intel(R) Thread Checker will ignore them. The reads are used in a double-check
// context, so the program is nonetheless correct despite the race.
return my_logical_size >= my_physical_size && my_physical_size < max_physical_size;
}
#endif//!TBB_NO_LEGACY
} // namespace internal
} // namespace tbb

View File

@@ -0,0 +1,235 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_concurrent_lru_cache_H
#define __TBB_concurrent_lru_cache_H
#if ! TBB_PREVIEW_CONCURRENT_LRU_CACHE
#error Set TBB_PREVIEW_CONCURRENT_LRU_CACHE to include concurrent_lru_cache.h
#endif
#include <map>
#include <list>
#include "tbb_stddef.h"
#include "atomic.h"
#include "internal/_aggregator_impl.h"
namespace tbb{
namespace interface6 {
template <typename key_type, typename value_type, typename value_functor_type = value_type (*)(key_type) >
class concurrent_lru_cache : internal::no_assign{
private:
typedef concurrent_lru_cache self_type;
typedef value_functor_type value_function_type;
typedef std::size_t ref_counter_type;
struct map_value_type;
typedef std::map<key_type, map_value_type> map_storage_type;
typedef std::list<typename map_storage_type::iterator> lru_list_type;
struct map_value_type {
value_type my_value;
ref_counter_type my_ref_counter;
typename lru_list_type::iterator my_lru_list_iterator;
bool my_is_ready;
map_value_type (value_type const& a_value, ref_counter_type a_ref_counter, typename lru_list_type::iterator a_lru_list_iterator, bool a_is_ready)
: my_value(a_value), my_ref_counter(a_ref_counter), my_lru_list_iterator (a_lru_list_iterator), my_is_ready(a_is_ready)
{}
};
class handle_object;
struct aggregator_operation;
typedef aggregator_operation aggregated_operation_type;
typedef tbb::internal::aggregating_functor<self_type,aggregated_operation_type> aggregator_function_type;
friend class tbb::internal::aggregating_functor<self_type,aggregated_operation_type>;
typedef tbb::internal::aggregator<aggregator_function_type, aggregated_operation_type> aggregator_type;
private:
value_function_type my_value_function;
std::size_t const my_number_of_lru_history_items;
map_storage_type my_map_storage;
lru_list_type my_lru_list;
aggregator_type my_aggregator;
public:
typedef handle_object handle;
public:
concurrent_lru_cache(value_function_type f, std::size_t number_of_lru_history_items)
: my_value_function(f),my_number_of_lru_history_items(number_of_lru_history_items)
{
my_aggregator.initialize_handler(aggregator_function_type(this));
}
handle_object operator[](key_type k){
retrieve_aggregator_operation op(k);
my_aggregator.execute(&op);
if (op.is_new_value_needed()){
op.result().second.my_value = my_value_function(k);
__TBB_store_with_release(op.result().second.my_is_ready, true);
}else{
tbb::internal::spin_wait_while_eq(op.result().second.my_is_ready,false);
}
return handle_object(*this,op.result());
}
private:
void signal_end_of_usage(typename map_storage_type::reference value_ref){
signal_end_of_usage_aggregator_operation op(value_ref);
my_aggregator.execute(&op);
}
private:
struct handle_move_t:no_assign{
concurrent_lru_cache & my_cache_ref;
typename map_storage_type::reference my_map_record_ref;
handle_move_t(concurrent_lru_cache & cache_ref, typename map_storage_type::reference value_ref):my_cache_ref(cache_ref),my_map_record_ref(value_ref) {};
};
class handle_object {
concurrent_lru_cache * my_cache_pointer;
typename map_storage_type::reference my_map_record_ref;
public:
handle_object(concurrent_lru_cache & cache_ref, typename map_storage_type::reference value_ref):my_cache_pointer(&cache_ref), my_map_record_ref(value_ref) {}
handle_object(handle_move_t m):my_cache_pointer(&m.my_cache_ref), my_map_record_ref(m.my_map_record_ref){}
operator handle_move_t(){ return move(*this);}
value_type& value(){
__TBB_ASSERT(my_cache_pointer,"get value from moved from object?");
return my_map_record_ref.second.my_value;
}
~handle_object(){
if (my_cache_pointer){
my_cache_pointer->signal_end_of_usage(my_map_record_ref);
}
}
private:
friend handle_move_t move(handle_object& h){
return handle_object::move(h);
}
static handle_move_t move(handle_object& h){
__TBB_ASSERT(h.my_cache_pointer,"move from the same object twice ?");
concurrent_lru_cache * cache_pointer = NULL;
std::swap(cache_pointer,h.my_cache_pointer);
return handle_move_t(*cache_pointer,h.my_map_record_ref);
}
private:
void operator=(handle_object&);
#if __SUNPRO_CC
// Presumably due to a compiler error, private copy constructor
// breaks expressions like handle h = cache[key];
public:
#endif
handle_object(handle_object &);
};
private:
//TODO: looks like aggregator_operation is a perfect match for statically typed variant type
struct aggregator_operation : tbb::internal::aggregated_operation<aggregator_operation>{
enum e_op_type {op_retive, op_signal_end_of_usage};
//TODO: try to use pointer to function apply_visitor here
//TODO: try virtual functions and measure the difference
e_op_type my_operation_type;
aggregator_operation(e_op_type operation_type): my_operation_type(operation_type) {}
void cast_and_handle(self_type& container ){
if (my_operation_type==op_retive){
static_cast<retrieve_aggregator_operation*>(this)->handle(container);
}else{
static_cast<signal_end_of_usage_aggregator_operation*>(this)->handle(container);
}
}
};
struct retrieve_aggregator_operation : aggregator_operation, private internal::no_assign {
key_type my_key;
typename map_storage_type::pointer my_result_map_record_pointer;
bool my_is_new_value_needed;
retrieve_aggregator_operation(key_type key):aggregator_operation(aggregator_operation::op_retive),my_key(key),my_is_new_value_needed(false){}
void handle(self_type& container ){
my_result_map_record_pointer = & container.retrieve_serial(my_key,my_is_new_value_needed);
}
typename map_storage_type::reference result(){ return * my_result_map_record_pointer; }
bool is_new_value_needed(){return my_is_new_value_needed;}
};
struct signal_end_of_usage_aggregator_operation : aggregator_operation, private internal::no_assign {
typename map_storage_type::reference my_map_record_ref;
signal_end_of_usage_aggregator_operation(typename map_storage_type::reference map_record_ref):aggregator_operation(aggregator_operation::op_signal_end_of_usage),my_map_record_ref(map_record_ref){}
void handle(self_type& container ){
container.signal_end_of_usage_serial(my_map_record_ref);
}
};
private:
void handle_operations(aggregator_operation* op_list){
while(op_list){
op_list->cast_and_handle(*this);
aggregator_operation* tmp = op_list;
op_list=op_list->next;
tbb::internal::itt_store_word_with_release(tmp->status, uintptr_t(1));
}
}
private:
typename map_storage_type::reference retrieve_serial(key_type k, bool& is_new_value_needed){
typename map_storage_type::iterator it = my_map_storage.find(k);
if (it == my_map_storage.end()){
it = my_map_storage.insert(it,std::make_pair(k,map_value_type(value_type(),0,my_lru_list.end(),false)));
is_new_value_needed = true;
}else {
typename lru_list_type::iterator list_it = it->second.my_lru_list_iterator;
if (list_it!=my_lru_list.end()) {
__TBB_ASSERT(!it->second.my_ref_counter,"item to be evicted should not have a live references");
//item is going to be used. Therefore it is not a subject for eviction
//so - remove it from LRU history.
my_lru_list.erase(list_it);
it->second.my_lru_list_iterator= my_lru_list.end();
}
}
++(it->second.my_ref_counter);
return *it;
}
void signal_end_of_usage_serial(typename map_storage_type::reference map_record_ref){
typename map_storage_type::iterator it = my_map_storage.find(map_record_ref.first);
__TBB_ASSERT(it!=my_map_storage.end(),"cache should not return past-end iterators to outer world");
__TBB_ASSERT(&(*it) == &map_record_ref,"dangling reference has been returned to outside world? data race ?");
__TBB_ASSERT( my_lru_list.end()== std::find(my_lru_list.begin(),my_lru_list.end(),it),
"object in use should not be in list of unused objects ");
if (! --(it->second.my_ref_counter)){
//it was the last reference so put it to the LRU history
if (my_lru_list.size()>=my_number_of_lru_history_items){
//evict items in order to get a space
size_t number_of_elements_to_evict = 1 + my_lru_list.size() - my_number_of_lru_history_items;
for (size_t i=0; i<number_of_elements_to_evict; ++i){
typename map_storage_type::iterator it_to_evict = my_lru_list.back();
__TBB_ASSERT(!it_to_evict->second.my_ref_counter,"item to be evicted should not have a live references");
my_lru_list.pop_back();
my_map_storage.erase(it_to_evict);
}
}
my_lru_list.push_front(it);
it->second.my_lru_list_iterator = my_lru_list.begin();
}
}
};
} // namespace interface6
using interface6::concurrent_lru_cache;
} // namespace tbb
#endif //__TBB_concurrent_lru_cache_H

View File

@@ -0,0 +1,137 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "concurrent_monitor.h"
namespace tbb {
namespace internal {
void concurrent_monitor::thread_context::init() {
new (sema.begin()) binary_semaphore;
ready = true;
}
concurrent_monitor::~concurrent_monitor() {
abort_all();
__TBB_ASSERT( waitset_ec.empty(), "waitset not empty?" );
}
void concurrent_monitor::prepare_wait( thread_context& thr, uintptr_t ctx ) {
if( !thr.ready )
thr.init();
// this is good place to pump previous spurious wakeup
else if( thr.spurious ) {
thr.spurious = false;
thr.semaphore().P();
}
thr.context = ctx;
thr.in_waitset = true;
{
tbb::spin_mutex::scoped_lock l( mutex_ec );
__TBB_store_relaxed( thr.epoch, __TBB_load_relaxed(epoch) );
waitset_ec.add( (waitset_t::node_t*)&thr );
}
atomic_fence();
}
void concurrent_monitor::cancel_wait( thread_context& thr ) {
// spurious wakeup will be pumped in the following prepare_wait()
thr.spurious = true;
// try to remove node from waitset
bool th_in_waitset = thr.in_waitset;
if( th_in_waitset ) {
tbb::spin_mutex::scoped_lock l( mutex_ec );
if (thr.in_waitset) {
// successfully removed from waitset,
// so there will be no spurious wakeup
thr.in_waitset = false;
thr.spurious = false;
waitset_ec.remove( (waitset_t::node_t&)thr );
}
}
}
void concurrent_monitor::notify_one_relaxed() {
if( waitset_ec.empty() )
return;
waitset_node_t* n;
const waitset_node_t* end = waitset_ec.end();
{
tbb::spin_mutex::scoped_lock l( mutex_ec );
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
n = waitset_ec.front();
if( n!=end ) {
waitset_ec.remove( *n );
to_thread_context(n)->in_waitset = false;
}
}
if( n!=end )
to_thread_context(n)->semaphore().V();
}
void concurrent_monitor::notify_all_relaxed() {
if( waitset_ec.empty() )
return;
dllist_t temp;
const waitset_node_t* end;
{
tbb::spin_mutex::scoped_lock l( mutex_ec );
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
waitset_ec.flush_to( temp );
end = temp.end();
for( waitset_node_t* n=temp.front(); n!=end; n=n->next )
to_thread_context(n)->in_waitset = false;
}
waitset_node_t* nxt;
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
nxt = n->next;
to_thread_context(n)->semaphore().V();
}
#if TBB_USE_ASSERT
temp.clear();
#endif
}
void concurrent_monitor::abort_all_relaxed() {
if( waitset_ec.empty() )
return;
dllist_t temp;
const waitset_node_t* end;
{
tbb::spin_mutex::scoped_lock l( mutex_ec );
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
waitset_ec.flush_to( temp );
end = temp.end();
for( waitset_node_t* n=temp.front(); n!=end; n=n->next )
to_thread_context(n)->in_waitset = false;
}
waitset_node_t* nxt;
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
nxt = n->next;
to_thread_context(n)->aborted = true;
to_thread_context(n)->semaphore().V();
}
#if TBB_USE_ASSERT
temp.clear();
#endif
}
} // namespace internal
} // namespace tbb

View File

@@ -0,0 +1,241 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_concurrent_monitor_H
#define __TBB_concurrent_monitor_H
#include "tbb/tbb_stddef.h"
#include "tbb/atomic.h"
#include "tbb/spin_mutex.h"
#include "tbb/tbb_exception.h"
#include "tbb/aligned_space.h"
#include "semaphore.h"
namespace tbb {
namespace internal {
//! Circular doubly-linked list with sentinel
/** head.next points to the front and head.prev points to the back */
class circular_doubly_linked_list_with_sentinel : no_copy {
public:
struct node_t {
node_t* next;
node_t* prev;
explicit node_t() : next((node_t*)(uintptr_t)0xcdcdcdcd), prev((node_t*)(uintptr_t)0xcdcdcdcd) {}
};
// ctor
circular_doubly_linked_list_with_sentinel() {clear();}
// dtor
~circular_doubly_linked_list_with_sentinel() {__TBB_ASSERT( head.next==&head && head.prev==&head, "the list is not empty" );}
inline size_t size() const {return count;}
inline bool empty() const {return size()==0;}
inline node_t* front() const {return head.next;}
inline node_t* last() const {return head.prev;}
inline node_t* begin() const {return front();}
inline const node_t* end() const {return &head;}
//! add to the back of the list
inline void add( node_t* n ) {
__TBB_store_relaxed(count, __TBB_load_relaxed(count) + 1);
n->prev = head.prev;
n->next = &head;
head.prev->next = n;
head.prev = n;
}
//! remove node 'n'
inline void remove( node_t& n ) {
__TBB_store_relaxed(count, __TBB_load_relaxed(count) - 1);
n.prev->next = n.next;
n.next->prev = n.prev;
}
//! move all elements to 'lst' and initialize the 'this' list
inline void flush_to( circular_doubly_linked_list_with_sentinel& lst ) {
if( const size_t l_count = __TBB_load_relaxed(count) ) {
__TBB_store_relaxed(lst.count, l_count);
lst.head.next = head.next;
lst.head.prev = head.prev;
head.next->prev = &lst.head;
head.prev->next = &lst.head;
clear();
}
}
void clear() {head.next = head.prev = &head; __TBB_store_relaxed(count, 0);}
private:
__TBB_atomic size_t count;
node_t head;
};
typedef circular_doubly_linked_list_with_sentinel waitset_t;
typedef circular_doubly_linked_list_with_sentinel dllist_t;
typedef circular_doubly_linked_list_with_sentinel::node_t waitset_node_t;
//! concurrent_monitor
/** fine-grained concurrent_monitor implementation */
class concurrent_monitor : no_copy {
public:
/** per-thread descriptor for concurrent_monitor */
class thread_context : waitset_node_t, no_copy {
friend class concurrent_monitor;
public:
thread_context() : spurious(false), aborted(false), ready(false), context(0) {
epoch = 0;
in_waitset = false;
}
~thread_context() {
if (ready) {
if( spurious ) semaphore().P();
semaphore().~binary_semaphore();
}
}
binary_semaphore& semaphore() { return *sema.begin(); }
private:
//! The method for lazy initialization of the thread_context's semaphore.
// Inlining of the method is undesirable, due to extra instructions for
// exception support added at caller side.
__TBB_NOINLINE( void init() );
tbb::aligned_space<binary_semaphore> sema;
__TBB_atomic unsigned epoch;
tbb::atomic<bool> in_waitset;
bool spurious;
bool aborted;
bool ready;
uintptr_t context;
};
//! ctor
concurrent_monitor() {__TBB_store_relaxed(epoch, 0);}
//! dtor
~concurrent_monitor() ;
//! prepare wait by inserting 'thr' into the wait queue
void prepare_wait( thread_context& thr, uintptr_t ctx = 0 );
//! Commit wait if event count has not changed; otherwise, cancel wait.
/** Returns true if committed, false if canceled. */
inline bool commit_wait( thread_context& thr ) {
const bool do_it = thr.epoch == __TBB_load_relaxed(epoch);
// this check is just an optimization
if( do_it ) {
__TBB_ASSERT( thr.ready, "use of commit_wait() without prior prepare_wait()");
thr.semaphore().P();
__TBB_ASSERT( !thr.in_waitset, "still in the queue?" );
if( thr.aborted )
throw_exception( eid_user_abort );
} else {
cancel_wait( thr );
}
return do_it;
}
//! Cancel the wait. Removes the thread from the wait queue if not removed yet.
void cancel_wait( thread_context& thr );
//! Wait for a condition to be satisfied with waiting-on context
template<typename WaitUntil, typename Context>
void wait( WaitUntil until, Context on );
//! Notify one thread about the event
void notify_one() {atomic_fence(); notify_one_relaxed();}
//! Notify one thread about the event. Relaxed version.
void notify_one_relaxed();
//! Notify all waiting threads of the event
void notify_all() {atomic_fence(); notify_all_relaxed();}
//! Notify all waiting threads of the event; Relaxed version
void notify_all_relaxed();
//! Notify waiting threads of the event that satisfies the given predicate
template<typename P> void notify( const P& predicate ) {atomic_fence(); notify_relaxed( predicate );}
//! Notify waiting threads of the event that satisfies the given predicate; Relaxed version
template<typename P> void notify_relaxed( const P& predicate );
//! Abort any sleeping threads at the time of the call
void abort_all() {atomic_fence(); abort_all_relaxed(); }
//! Abort any sleeping threads at the time of the call; Relaxed version
void abort_all_relaxed();
private:
tbb::spin_mutex mutex_ec;
waitset_t waitset_ec;
__TBB_atomic unsigned epoch;
thread_context* to_thread_context( waitset_node_t* n ) { return static_cast<thread_context*>(n); }
};
template<typename WaitUntil, typename Context>
void concurrent_monitor::wait( WaitUntil until, Context on )
{
bool slept = false;
thread_context thr_ctx;
prepare_wait( thr_ctx, on() );
while( !until() ) {
if( (slept = commit_wait( thr_ctx ) )==true )
if( until() ) break;
slept = false;
prepare_wait( thr_ctx, on() );
}
if( !slept )
cancel_wait( thr_ctx );
}
template<typename P>
void concurrent_monitor::notify_relaxed( const P& predicate ) {
if( waitset_ec.empty() )
return;
dllist_t temp;
waitset_node_t* nxt;
const waitset_node_t* end = waitset_ec.end();
{
tbb::spin_mutex::scoped_lock l( mutex_ec );
__TBB_store_relaxed(epoch, __TBB_load_relaxed(epoch) + 1);
for( waitset_node_t* n=waitset_ec.last(); n!=end; n=nxt ) {
nxt = n->prev;
thread_context* thr = to_thread_context( n );
if( predicate( thr->context ) ) {
waitset_ec.remove( *n );
thr->in_waitset = false;
temp.add( n );
}
}
}
end = temp.end();
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
nxt = n->next;
to_thread_context(n)->semaphore().V();
}
#if TBB_USE_ASSERT
temp.clear();
#endif
}
} // namespace internal
} // namespace tbb
#endif /* __TBB_concurrent_monitor_H */

View File

@@ -0,0 +1,457 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_concurrent_priority_queue_H
#define __TBB_concurrent_priority_queue_H
#include "atomic.h"
#include "cache_aligned_allocator.h"
#include "tbb_exception.h"
#include "tbb_stddef.h"
#include "tbb_profiling.h"
#include "internal/_aggregator_impl.h"
#include <vector>
#include <iterator>
#include <functional>
#if __TBB_INITIALIZER_LISTS_PRESENT
#include <initializer_list>
#endif
namespace tbb {
namespace interface5 {
using namespace tbb::internal;
//! Concurrent priority queue
template <typename T, typename Compare=std::less<T>, typename A=cache_aligned_allocator<T> >
class concurrent_priority_queue {
public:
//! Element type in the queue.
typedef T value_type;
//! Reference type
typedef T& reference;
//! Const reference type
typedef const T& const_reference;
//! Integral type for representing size of the queue.
typedef size_t size_type;
//! Difference type for iterator
typedef ptrdiff_t difference_type;
//! Allocator type
typedef A allocator_type;
//! Constructs a new concurrent_priority_queue with default capacity
explicit concurrent_priority_queue(const allocator_type& a = allocator_type()) : mark(0), my_size(0), data(a)
{
my_aggregator.initialize_handler(my_functor_t(this));
}
//! Constructs a new concurrent_priority_queue with init_sz capacity
explicit concurrent_priority_queue(size_type init_capacity, const allocator_type& a = allocator_type()) :
mark(0), my_size(0), data(a)
{
data.reserve(init_capacity);
my_aggregator.initialize_handler(my_functor_t(this));
}
//! [begin,end) constructor
template<typename InputIterator>
concurrent_priority_queue(InputIterator begin, InputIterator end, const allocator_type& a = allocator_type()) :
mark(0), data(begin, end, a)
{
my_aggregator.initialize_handler(my_functor_t(this));
heapify();
my_size = data.size();
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Constructor from std::initializer_list
concurrent_priority_queue(std::initializer_list<T> init_list, const allocator_type &a = allocator_type()) :
mark(0),data(init_list.begin(), init_list.end(), a)
{
my_aggregator.initialize_handler(my_functor_t(this));
heapify();
my_size = data.size();
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
//! Copy constructor
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
explicit concurrent_priority_queue(const concurrent_priority_queue& src) : mark(src.mark),
my_size(src.my_size), data(src.data.begin(), src.data.end(), src.data.get_allocator())
{
my_aggregator.initialize_handler(my_functor_t(this));
heapify();
}
//! Copy constructor with specific allocator
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
concurrent_priority_queue(const concurrent_priority_queue& src, const allocator_type& a) : mark(src.mark),
my_size(src.my_size), data(src.data.begin(), src.data.end(), a)
{
my_aggregator.initialize_handler(my_functor_t(this));
heapify();
}
//! Assignment operator
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
concurrent_priority_queue& operator=(const concurrent_priority_queue& src) {
if (this != &src) {
vector_t(src.data.begin(), src.data.end(), src.data.get_allocator()).swap(data);
mark = src.mark;
my_size = src.my_size;
}
return *this;
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Move constructor
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
concurrent_priority_queue(concurrent_priority_queue&& src) : mark(src.mark),
my_size(src.my_size), data(std::move(src.data))
{
my_aggregator.initialize_handler(my_functor_t(this));
}
//! Move constructor with specific allocator
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
concurrent_priority_queue(concurrent_priority_queue&& src, const allocator_type& a) : mark(src.mark),
my_size(src.my_size),
#if __TBB_ALLOCATOR_TRAITS_PRESENT
data(std::move(src.data), a)
#else
// Some early version of C++11 STL vector does not have a constructor of vector(vector&& , allocator).
// It seems that the reason is absence of support of allocator_traits (stateful allocators).
data(a)
#endif //__TBB_ALLOCATOR_TRAITS_PRESENT
{
my_aggregator.initialize_handler(my_functor_t(this));
#if !__TBB_ALLOCATOR_TRAITS_PRESENT
if (a != src.data.get_allocator()){
data.reserve(src.data.size());
data.assign(std::make_move_iterator(src.data.begin()), std::make_move_iterator(src.data.end()));
}else{
data = std::move(src.data);
}
#endif //!__TBB_ALLOCATOR_TRAITS_PRESENT
}
//! Move assignment operator
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
concurrent_priority_queue& operator=( concurrent_priority_queue&& src) {
if (this != &src) {
mark = src.mark;
my_size = src.my_size;
#if !__TBB_ALLOCATOR_TRAITS_PRESENT
if (data.get_allocator() != src.data.get_allocator()){
vector_t(std::make_move_iterator(src.data.begin()), std::make_move_iterator(src.data.end()), data.get_allocator()).swap(data);
}else
#endif //!__TBB_ALLOCATOR_TRAITS_PRESENT
{
data = std::move(src.data);
}
}
return *this;
}
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
//! Assign the queue from [begin,end) range, not thread-safe
template<typename InputIterator>
void assign(InputIterator begin, InputIterator end) {
vector_t(begin, end, data.get_allocator()).swap(data);
mark = 0;
my_size = data.size();
heapify();
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Assign the queue from std::initializer_list, not thread-safe
void assign(std::initializer_list<T> il) { this->assign(il.begin(), il.end()); }
//! Assign from std::initializer_list, not thread-safe
concurrent_priority_queue& operator=(std::initializer_list<T> il) {
this->assign(il.begin(), il.end());
return *this;
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
//! Returns true if empty, false otherwise
/** Returned value may not reflect results of pending operations.
This operation reads shared data and will trigger a race condition. */
bool empty() const { return size()==0; }
//! Returns the current number of elements contained in the queue
/** Returned value may not reflect results of pending operations.
This operation reads shared data and will trigger a race condition. */
size_type size() const { return __TBB_load_with_acquire(my_size); }
//! Pushes elem onto the queue, increasing capacity of queue if necessary
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
void push(const_reference elem) {
cpq_operation op_data(elem, PUSH_OP);
my_aggregator.execute(&op_data);
if (op_data.status == FAILED) // exception thrown
throw_exception(eid_bad_alloc);
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Pushes elem onto the queue, increasing capacity of queue if necessary
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
void push(value_type &&elem) {
cpq_operation op_data(elem, PUSH_RVALUE_OP);
my_aggregator.execute(&op_data);
if (op_data.status == FAILED) // exception thrown
throw_exception(eid_bad_alloc);
}
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
//! Constructs a new element using args as the arguments for its construction and pushes it onto the queue */
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
template<typename... Args>
void emplace(Args&&... args) {
push(value_type(std::forward<Args>(args)...));
}
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! Gets a reference to and removes highest priority element
/** If a highest priority element was found, sets elem and returns true,
otherwise returns false.
This operation can be safely used concurrently with other push, try_pop or emplace operations. */
bool try_pop(reference elem) {
cpq_operation op_data(POP_OP);
op_data.elem = &elem;
my_aggregator.execute(&op_data);
return op_data.status==SUCCEEDED;
}
//! Clear the queue; not thread-safe
/** This operation is unsafe if there are pending concurrent operations on the queue.
Resets size, effectively emptying queue; does not free space.
May not clear elements added in pending operations. */
void clear() {
data.clear();
mark = 0;
my_size = 0;
}
//! Swap this queue with another; not thread-safe
/** This operation is unsafe if there are pending concurrent operations on the queue. */
void swap(concurrent_priority_queue& q) {
using std::swap;
data.swap(q.data);
swap(mark, q.mark);
swap(my_size, q.my_size);
}
//! Return allocator object
allocator_type get_allocator() const { return data.get_allocator(); }
private:
enum operation_type {INVALID_OP, PUSH_OP, POP_OP, PUSH_RVALUE_OP};
enum operation_status { WAIT=0, SUCCEEDED, FAILED };
class cpq_operation : public aggregated_operation<cpq_operation> {
public:
operation_type type;
union {
value_type *elem;
size_type sz;
};
cpq_operation(const_reference e, operation_type t) :
type(t), elem(const_cast<value_type*>(&e)) {}
cpq_operation(operation_type t) : type(t) {}
};
class my_functor_t {
concurrent_priority_queue<T, Compare, A> *cpq;
public:
my_functor_t() {}
my_functor_t(concurrent_priority_queue<T, Compare, A> *cpq_) : cpq(cpq_) {}
void operator()(cpq_operation* op_list) {
cpq->handle_operations(op_list);
}
};
typedef tbb::internal::aggregator< my_functor_t, cpq_operation > aggregator_t;
aggregator_t my_aggregator;
//! Padding added to avoid false sharing
char padding1[NFS_MaxLineSize - sizeof(aggregator_t)];
//! The point at which unsorted elements begin
size_type mark;
__TBB_atomic size_type my_size;
Compare compare;
//! Padding added to avoid false sharing
char padding2[NFS_MaxLineSize - (2*sizeof(size_type)) - sizeof(Compare)];
//! Storage for the heap of elements in queue, plus unheapified elements
/** data has the following structure:
binary unheapified
heap elements
____|_______|____
| | |
v v v
[_|...|_|_|...|_| |...| ]
0 ^ ^ ^
| | |__capacity
| |__my_size
|__mark
Thus, data stores the binary heap starting at position 0 through
mark-1 (it may be empty). Then there are 0 or more elements
that have not yet been inserted into the heap, in positions
mark through my_size-1. */
typedef std::vector<value_type, allocator_type> vector_t;
vector_t data;
void handle_operations(cpq_operation *op_list) {
cpq_operation *tmp, *pop_list=NULL;
__TBB_ASSERT(mark == data.size(), NULL);
// First pass processes all constant (amortized; reallocation may happen) time pushes and pops.
while (op_list) {
// ITT note: &(op_list->status) tag is used to cover accesses to op_list
// node. This thread is going to handle the operation, and so will acquire it
// and perform the associated operation w/o triggering a race condition; the
// thread that created the operation is waiting on the status field, so when
// this thread is done with the operation, it will perform a
// store_with_release to give control back to the waiting thread in
// aggregator::insert_operation.
call_itt_notify(acquired, &(op_list->status));
__TBB_ASSERT(op_list->type != INVALID_OP, NULL);
tmp = op_list;
op_list = itt_hide_load_word(op_list->next);
if (tmp->type == POP_OP) {
if (mark < data.size() &&
compare(data[0], data[data.size()-1])) {
// there are newly pushed elems and the last one
// is higher than top
*(tmp->elem) = move(data[data.size()-1]);
__TBB_store_with_release(my_size, my_size-1);
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
data.pop_back();
__TBB_ASSERT(mark<=data.size(), NULL);
}
else { // no convenient item to pop; postpone
itt_hide_store_word(tmp->next, pop_list);
pop_list = tmp;
}
} else { // PUSH_OP or PUSH_RVALUE_OP
__TBB_ASSERT(tmp->type == PUSH_OP || tmp->type == PUSH_RVALUE_OP, "Unknown operation" );
__TBB_TRY{
if (tmp->type == PUSH_OP) {
data.push_back(*(tmp->elem));
} else {
data.push_back(move(*(tmp->elem)));
}
__TBB_store_with_release(my_size, my_size + 1);
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
} __TBB_CATCH(...) {
itt_store_word_with_release(tmp->status, uintptr_t(FAILED));
}
}
}
// second pass processes pop operations
while (pop_list) {
tmp = pop_list;
pop_list = itt_hide_load_word(pop_list->next);
__TBB_ASSERT(tmp->type == POP_OP, NULL);
if (data.empty()) {
itt_store_word_with_release(tmp->status, uintptr_t(FAILED));
}
else {
__TBB_ASSERT(mark<=data.size(), NULL);
if (mark < data.size() &&
compare(data[0], data[data.size()-1])) {
// there are newly pushed elems and the last one is
// higher than top
*(tmp->elem) = move(data[data.size()-1]);
__TBB_store_with_release(my_size, my_size-1);
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
data.pop_back();
}
else { // extract top and push last element down heap
*(tmp->elem) = move(data[0]);
__TBB_store_with_release(my_size, my_size-1);
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
reheap();
}
}
}
// heapify any leftover pushed elements before doing the next
// batch of operations
if (mark<data.size()) heapify();
__TBB_ASSERT(mark == data.size(), NULL);
}
//! Merge unsorted elements into heap
void heapify() {
if (!mark && data.size()>0) mark = 1;
for (; mark<data.size(); ++mark) {
// for each unheapified element under size
size_type cur_pos = mark;
value_type to_place = move(data[mark]);
do { // push to_place up the heap
size_type parent = (cur_pos-1)>>1;
if (!compare(data[parent], to_place)) break;
data[cur_pos] = move(data[parent]);
cur_pos = parent;
} while( cur_pos );
data[cur_pos] = move(to_place);
}
}
//! Re-heapify after an extraction
/** Re-heapify by pushing last element down the heap from the root. */
void reheap() {
size_type cur_pos=0, child=1;
while (child < mark) {
size_type target = child;
if (child+1 < mark && compare(data[child], data[child+1]))
++target;
// target now has the higher priority child
if (compare(data[target], data[data.size()-1])) break;
data[cur_pos] = move(data[target]);
cur_pos = target;
child = (cur_pos<<1)+1;
}
if (cur_pos != data.size()-1)
data[cur_pos] = move(data[data.size()-1]);
data.pop_back();
if (mark > data.size()) mark = data.size();
}
};
} // namespace interface5
using interface5::concurrent_priority_queue;
} // namespace tbb
#endif /* __TBB_concurrent_priority_queue_H */

View File

@@ -0,0 +1,670 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_stddef.h"
#include "tbb/tbb_machine.h"
#include "tbb/tbb_exception.h"
// Define required to satisfy test in internal file.
#define __TBB_concurrent_queue_H
#include "tbb/internal/_concurrent_queue_impl.h"
#include "concurrent_monitor.h"
#include "itt_notify.h"
#include <new>
#if !TBB_USE_EXCEPTIONS && _MSC_VER
// Suppress "C++ exception handler used, but unwind semantics are not enabled" warning in STL headers
#pragma warning (push)
#pragma warning (disable: 4530)
#endif
#include <cstring> // for memset()
#if !TBB_USE_EXCEPTIONS && _MSC_VER
#pragma warning (pop)
#endif
using namespace std;
#if defined(_MSC_VER) && defined(_Wp64)
// Workaround for overzealous compiler warnings in /Wp64 mode
#pragma warning (disable: 4267)
#endif
#define RECORD_EVENTS 0
namespace tbb {
namespace internal {
typedef concurrent_queue_base_v3 concurrent_queue_base;
typedef size_t ticket;
//! A queue using simple locking.
/** For efficiency, this class has no constructor.
The caller is expected to zero-initialize it. */
struct micro_queue {
typedef concurrent_queue_base::page page;
friend class micro_queue_pop_finalizer;
atomic<page*> head_page;
atomic<ticket> head_counter;
atomic<page*> tail_page;
atomic<ticket> tail_counter;
spin_mutex page_mutex;
void push( const void* item, ticket k, concurrent_queue_base& base,
concurrent_queue_base::copy_specifics op_type );
void abort_push( ticket k, concurrent_queue_base& base );
bool pop( void* dst, ticket k, concurrent_queue_base& base );
micro_queue& assign( const micro_queue& src, concurrent_queue_base& base,
concurrent_queue_base::copy_specifics op_type );
page* make_copy ( concurrent_queue_base& base, const page* src_page, size_t begin_in_page,
size_t end_in_page, ticket& g_index, concurrent_queue_base::copy_specifics op_type ) ;
void make_invalid( ticket k );
};
// we need to yank it out of micro_queue because of concurrent_queue_base::deallocate_page being virtual.
class micro_queue_pop_finalizer: no_copy {
typedef concurrent_queue_base::page page;
ticket my_ticket;
micro_queue& my_queue;
page* my_page;
concurrent_queue_base &base;
public:
micro_queue_pop_finalizer( micro_queue& queue, concurrent_queue_base& b, ticket k, page* p ) :
my_ticket(k), my_queue(queue), my_page(p), base(b)
{}
~micro_queue_pop_finalizer() {
page* p = my_page;
if( p ) {
spin_mutex::scoped_lock lock( my_queue.page_mutex );
page* q = p->next;
my_queue.head_page = q;
if( !q ) {
my_queue.tail_page = NULL;
}
}
my_queue.head_counter = my_ticket;
if( p )
base.deallocate_page( p );
}
};
struct predicate_leq {
ticket t;
predicate_leq( ticket t_ ) : t(t_) {}
bool operator() ( uintptr_t p ) const {return (ticket)p<=t;}
};
//! Internal representation of a ConcurrentQueue.
/** For efficiency, this class has no constructor.
The caller is expected to zero-initialize it. */
class concurrent_queue_rep {
public:
private:
friend struct micro_queue;
//! Approximately n_queue/golden ratio
static const size_t phi = 3;
public:
//! Must be power of 2
static const size_t n_queue = 8;
//! Map ticket to an array index
static size_t index( ticket k ) {
return k*phi%n_queue;
}
atomic<ticket> head_counter;
concurrent_monitor items_avail;
atomic<size_t> n_invalid_entries;
char pad1[NFS_MaxLineSize-((sizeof(atomic<ticket>)+sizeof(concurrent_monitor)+sizeof(atomic<size_t>))&(NFS_MaxLineSize-1))];
atomic<ticket> tail_counter;
concurrent_monitor slots_avail;
char pad2[NFS_MaxLineSize-((sizeof(atomic<ticket>)+sizeof(concurrent_monitor))&(NFS_MaxLineSize-1))];
micro_queue array[n_queue];
micro_queue& choose( ticket k ) {
// The formula here approximates LRU in a cache-oblivious way.
return array[index(k)];
}
//! Value for effective_capacity that denotes unbounded queue.
static const ptrdiff_t infinite_capacity = ptrdiff_t(~size_t(0)/2);
};
#if _MSC_VER && !defined(__INTEL_COMPILER)
// unary minus operator applied to unsigned type, result still unsigned
#pragma warning( push )
#pragma warning( disable: 4146 )
#endif
static void* invalid_page;
//------------------------------------------------------------------------
// micro_queue
//------------------------------------------------------------------------
void micro_queue::push( const void* item, ticket k, concurrent_queue_base& base,
concurrent_queue_base::copy_specifics op_type ) {
k &= -concurrent_queue_rep::n_queue;
page* p = NULL;
// find index on page where we would put the data
size_t index = modulo_power_of_two( k/concurrent_queue_rep::n_queue, base.items_per_page );
if( !index ) { // make a new page
__TBB_TRY {
p = base.allocate_page();
} __TBB_CATCH(...) {
++base.my_rep->n_invalid_entries;
make_invalid( k );
}
p->mask = 0;
p->next = NULL;
}
// wait for my turn
if( tail_counter!=k ) // The developer insisted on keeping first check out of the backoff loop
for( atomic_backoff b(true);;b.pause() ) {
ticket tail = tail_counter;
if( tail==k ) break;
else if( tail&0x1 ) {
// no memory. throws an exception; assumes concurrent_queue_rep::n_queue>1
++base.my_rep->n_invalid_entries;
throw_exception( eid_bad_last_alloc );
}
}
if( p ) { // page is newly allocated; insert in micro_queue
spin_mutex::scoped_lock lock( page_mutex );
if( page* q = tail_page )
q->next = p;
else
head_page = p;
tail_page = p;
}
if (item) {
p = tail_page;
ITT_NOTIFY( sync_acquired, p );
__TBB_TRY {
if( concurrent_queue_base::copy == op_type ) {
base.copy_item( *p, index, item );
} else {
__TBB_ASSERT( concurrent_queue_base::move == op_type, NULL );
static_cast<concurrent_queue_base_v8&>(base).move_item( *p, index, item );
}
} __TBB_CATCH(...) {
++base.my_rep->n_invalid_entries;
tail_counter += concurrent_queue_rep::n_queue;
__TBB_RETHROW();
}
ITT_NOTIFY( sync_releasing, p );
// If no exception was thrown, mark item as present.
p->mask |= uintptr_t(1)<<index;
}
else // no item; this was called from abort_push
++base.my_rep->n_invalid_entries;
tail_counter += concurrent_queue_rep::n_queue;
}
void micro_queue::abort_push( ticket k, concurrent_queue_base& base ) {
push(NULL, k, base, concurrent_queue_base::copy);
}
bool micro_queue::pop( void* dst, ticket k, concurrent_queue_base& base ) {
k &= -concurrent_queue_rep::n_queue;
spin_wait_until_eq( head_counter, k );
spin_wait_while_eq( tail_counter, k );
page& p = *head_page;
__TBB_ASSERT( &p, NULL );
size_t index = modulo_power_of_two( k/concurrent_queue_rep::n_queue, base.items_per_page );
bool success = false;
{
micro_queue_pop_finalizer finalizer( *this, base, k+concurrent_queue_rep::n_queue, index==base.items_per_page-1 ? &p : NULL );
if( p.mask & uintptr_t(1)<<index ) {
success = true;
ITT_NOTIFY( sync_acquired, dst );
ITT_NOTIFY( sync_acquired, head_page );
base.assign_and_destroy_item( dst, p, index );
ITT_NOTIFY( sync_releasing, head_page );
} else {
--base.my_rep->n_invalid_entries;
}
}
return success;
}
micro_queue& micro_queue::assign( const micro_queue& src, concurrent_queue_base& base,
concurrent_queue_base::copy_specifics op_type )
{
head_counter = src.head_counter;
tail_counter = src.tail_counter;
const page* srcp = src.head_page;
if( srcp ) {
ticket g_index = head_counter;
__TBB_TRY {
size_t n_items = (tail_counter-head_counter)/concurrent_queue_rep::n_queue;
size_t index = modulo_power_of_two( head_counter/concurrent_queue_rep::n_queue, base.items_per_page );
size_t end_in_first_page = (index+n_items<base.items_per_page)?(index+n_items):base.items_per_page;
head_page = make_copy( base, srcp, index, end_in_first_page, g_index, op_type );
page* cur_page = head_page;
if( srcp != src.tail_page ) {
for( srcp = srcp->next; srcp!=src.tail_page; srcp=srcp->next ) {
cur_page->next = make_copy( base, srcp, 0, base.items_per_page, g_index, op_type );
cur_page = cur_page->next;
}
__TBB_ASSERT( srcp==src.tail_page, NULL );
size_t last_index = modulo_power_of_two( tail_counter/concurrent_queue_rep::n_queue, base.items_per_page );
if( last_index==0 ) last_index = base.items_per_page;
cur_page->next = make_copy( base, srcp, 0, last_index, g_index, op_type );
cur_page = cur_page->next;
}
tail_page = cur_page;
} __TBB_CATCH(...) {
make_invalid( g_index );
}
} else {
head_page = tail_page = NULL;
}
return *this;
}
concurrent_queue_base::page* micro_queue::make_copy( concurrent_queue_base& base,
const concurrent_queue_base::page* src_page, size_t begin_in_page, size_t end_in_page,
ticket& g_index, concurrent_queue_base::copy_specifics op_type )
{
page* new_page = base.allocate_page();
new_page->next = NULL;
new_page->mask = src_page->mask;
for( ; begin_in_page!=end_in_page; ++begin_in_page, ++g_index )
if( new_page->mask & uintptr_t(1)<<begin_in_page )
if( concurrent_queue_base::copy == op_type ) {
base.copy_page_item( *new_page, begin_in_page, *src_page, begin_in_page );
} else {
__TBB_ASSERT( concurrent_queue_base::move == op_type, NULL );
static_cast<concurrent_queue_base_v8&>(base).move_page_item( *new_page, begin_in_page, *src_page, begin_in_page );
}
return new_page;
}
void micro_queue::make_invalid( ticket k )
{
static concurrent_queue_base::page dummy = {static_cast<page*>((void*)1), 0};
// mark it so that no more pushes are allowed.
invalid_page = &dummy;
{
spin_mutex::scoped_lock lock( page_mutex );
tail_counter = k+concurrent_queue_rep::n_queue+1;
if( page* q = tail_page )
q->next = static_cast<page*>(invalid_page);
else
head_page = static_cast<page*>(invalid_page);
tail_page = static_cast<page*>(invalid_page);
}
__TBB_RETHROW();
}
#if _MSC_VER && !defined(__INTEL_COMPILER)
#pragma warning( pop )
#endif // warning 4146 is back
//------------------------------------------------------------------------
// concurrent_queue_base
//------------------------------------------------------------------------
concurrent_queue_base_v3::concurrent_queue_base_v3( size_t item_sz ) {
items_per_page = item_sz<= 8 ? 32 :
item_sz<= 16 ? 16 :
item_sz<= 32 ? 8 :
item_sz<= 64 ? 4 :
item_sz<=128 ? 2 :
1;
my_capacity = size_t(-1)/(item_sz>1 ? item_sz : 2);
my_rep = cache_aligned_allocator<concurrent_queue_rep>().allocate(1);
__TBB_ASSERT( (size_t)my_rep % NFS_GetLineSize()==0, "alignment error" );
__TBB_ASSERT( (size_t)&my_rep->head_counter % NFS_GetLineSize()==0, "alignment error" );
__TBB_ASSERT( (size_t)&my_rep->tail_counter % NFS_GetLineSize()==0, "alignment error" );
__TBB_ASSERT( (size_t)&my_rep->array % NFS_GetLineSize()==0, "alignment error" );
memset(my_rep,0,sizeof(concurrent_queue_rep));
new ( &my_rep->items_avail ) concurrent_monitor();
new ( &my_rep->slots_avail ) concurrent_monitor();
this->item_size = item_sz;
}
concurrent_queue_base_v3::~concurrent_queue_base_v3() {
size_t nq = my_rep->n_queue;
for( size_t i=0; i<nq; i++ )
__TBB_ASSERT( my_rep->array[i].tail_page==NULL, "pages were not freed properly" );
cache_aligned_allocator<concurrent_queue_rep>().deallocate(my_rep,1);
}
void concurrent_queue_base_v3::internal_push( const void* src ) {
internal_insert_item( src, copy );
}
void concurrent_queue_base_v8::internal_push_move( const void* src ) {
internal_insert_item( src, move );
}
void concurrent_queue_base_v3::internal_insert_item( const void* src, copy_specifics op_type ) {
concurrent_queue_rep& r = *my_rep;
ticket k = r.tail_counter++;
ptrdiff_t e = my_capacity;
#if DO_ITT_NOTIFY
bool sync_prepare_done = false;
#endif
if( (ptrdiff_t)(k-r.head_counter)>=e ) { // queue is full
#if DO_ITT_NOTIFY
if( !sync_prepare_done ) {
ITT_NOTIFY( sync_prepare, &sync_prepare_done );
sync_prepare_done = true;
}
#endif
bool slept = false;
concurrent_monitor::thread_context thr_ctx;
r.slots_avail.prepare_wait( thr_ctx, ((ptrdiff_t)(k-e)) );
while( (ptrdiff_t)(k-r.head_counter)>=const_cast<volatile ptrdiff_t&>(e = my_capacity) ) {
__TBB_TRY {
slept = r.slots_avail.commit_wait( thr_ctx );
} __TBB_CATCH( tbb::user_abort& ) {
r.choose(k).abort_push(k, *this);
__TBB_RETHROW();
} __TBB_CATCH(...) {
__TBB_RETHROW();
}
if (slept == true) break;
r.slots_avail.prepare_wait( thr_ctx, ((ptrdiff_t)(k-e)) );
}
if( !slept )
r.slots_avail.cancel_wait( thr_ctx );
}
ITT_NOTIFY( sync_acquired, &sync_prepare_done );
__TBB_ASSERT( (ptrdiff_t)(k-r.head_counter)<my_capacity, NULL);
r.choose( k ).push( src, k, *this, op_type );
r.items_avail.notify( predicate_leq(k) );
}
void concurrent_queue_base_v3::internal_pop( void* dst ) {
concurrent_queue_rep& r = *my_rep;
ticket k;
#if DO_ITT_NOTIFY
bool sync_prepare_done = false;
#endif
do {
k=r.head_counter++;
if ( (ptrdiff_t)(r.tail_counter-k)<=0 ) { // queue is empty
#if DO_ITT_NOTIFY
if( !sync_prepare_done ) {
ITT_NOTIFY( sync_prepare, dst );
sync_prepare_done = true;
}
#endif
bool slept = false;
concurrent_monitor::thread_context thr_ctx;
r.items_avail.prepare_wait( thr_ctx, k );
while( (ptrdiff_t)(r.tail_counter-k)<=0 ) {
__TBB_TRY {
slept = r.items_avail.commit_wait( thr_ctx );
} __TBB_CATCH( tbb::user_abort& ) {
r.head_counter--;
__TBB_RETHROW();
} __TBB_CATCH(...) {
__TBB_RETHROW();
}
if (slept == true) break;
r.items_avail.prepare_wait( thr_ctx, k );
}
if( !slept )
r.items_avail.cancel_wait( thr_ctx );
}
__TBB_ASSERT((ptrdiff_t)(r.tail_counter-k)>0, NULL);
} while( !r.choose(k).pop(dst,k,*this) );
// wake up a producer..
r.slots_avail.notify( predicate_leq(k) );
}
void concurrent_queue_base_v3::internal_abort() {
concurrent_queue_rep& r = *my_rep;
r.items_avail.abort_all();
r.slots_avail.abort_all();
}
bool concurrent_queue_base_v3::internal_pop_if_present( void* dst ) {
concurrent_queue_rep& r = *my_rep;
ticket k;
do {
k = r.head_counter;
for(;;) {
if( (ptrdiff_t)(r.tail_counter-k)<=0 ) {
// Queue is empty
return false;
}
// Queue had item with ticket k when we looked. Attempt to get that item.
ticket tk=k;
k = r.head_counter.compare_and_swap( tk+1, tk );
if( k==tk )
break;
// Another thread snatched the item, retry.
}
} while( !r.choose( k ).pop( dst, k, *this ) );
r.slots_avail.notify( predicate_leq(k) );
return true;
}
bool concurrent_queue_base_v3::internal_push_if_not_full( const void* src ) {
return internal_insert_if_not_full( src, copy );
}
bool concurrent_queue_base_v8::internal_push_move_if_not_full( const void* src ) {
return internal_insert_if_not_full( src, move );
}
bool concurrent_queue_base_v3::internal_insert_if_not_full( const void* src, copy_specifics op_type ) {
concurrent_queue_rep& r = *my_rep;
ticket k = r.tail_counter;
for(;;) {
if( (ptrdiff_t)(k-r.head_counter)>=my_capacity ) {
// Queue is full
return false;
}
// Queue had empty slot with ticket k when we looked. Attempt to claim that slot.
ticket tk=k;
k = r.tail_counter.compare_and_swap( tk+1, tk );
if( k==tk )
break;
// Another thread claimed the slot, so retry.
}
r.choose(k).push(src, k, *this, op_type);
r.items_avail.notify( predicate_leq(k) );
return true;
}
ptrdiff_t concurrent_queue_base_v3::internal_size() const {
__TBB_ASSERT( sizeof(ptrdiff_t)<=sizeof(size_t), NULL );
return ptrdiff_t(my_rep->tail_counter-my_rep->head_counter-my_rep->n_invalid_entries);
}
bool concurrent_queue_base_v3::internal_empty() const {
ticket tc = my_rep->tail_counter;
ticket hc = my_rep->head_counter;
// if tc!=r.tail_counter, the queue was not empty at some point between the two reads.
return ( tc==my_rep->tail_counter && ptrdiff_t(tc-hc-my_rep->n_invalid_entries)<=0 );
}
void concurrent_queue_base_v3::internal_set_capacity( ptrdiff_t capacity, size_t /*item_sz*/ ) {
my_capacity = capacity<0 ? concurrent_queue_rep::infinite_capacity : capacity;
}
void concurrent_queue_base_v3::internal_finish_clear() {
size_t nq = my_rep->n_queue;
for( size_t i=0; i<nq; ++i ) {
page* tp = my_rep->array[i].tail_page;
__TBB_ASSERT( my_rep->array[i].head_page==tp, "at most one page should remain" );
if( tp!=NULL) {
if( tp!=invalid_page ) deallocate_page( tp );
my_rep->array[i].tail_page = NULL;
}
}
}
void concurrent_queue_base_v3::internal_throw_exception() const {
throw_exception( eid_bad_alloc );
}
void concurrent_queue_base_v3::internal_assign( const concurrent_queue_base& src, copy_specifics op_type ) {
items_per_page = src.items_per_page;
my_capacity = src.my_capacity;
// copy concurrent_queue_rep.
my_rep->head_counter = src.my_rep->head_counter;
my_rep->tail_counter = src.my_rep->tail_counter;
my_rep->n_invalid_entries = src.my_rep->n_invalid_entries;
// copy micro_queues
for( size_t i = 0; i<my_rep->n_queue; ++i )
my_rep->array[i].assign( src.my_rep->array[i], *this, op_type );
__TBB_ASSERT( my_rep->head_counter==src.my_rep->head_counter && my_rep->tail_counter==src.my_rep->tail_counter,
"the source concurrent queue should not be concurrently modified." );
}
void concurrent_queue_base_v3::assign( const concurrent_queue_base& src ) {
internal_assign( src, copy );
}
void concurrent_queue_base_v8::move_content( concurrent_queue_base_v8& src ) {
internal_assign( src, move );
}
//------------------------------------------------------------------------
// concurrent_queue_iterator_rep
//------------------------------------------------------------------------
class concurrent_queue_iterator_rep: no_assign {
public:
ticket head_counter;
const concurrent_queue_base& my_queue;
const size_t offset_of_last;
concurrent_queue_base::page* array[concurrent_queue_rep::n_queue];
concurrent_queue_iterator_rep( const concurrent_queue_base& queue, size_t offset_of_last_ ) :
head_counter(queue.my_rep->head_counter),
my_queue(queue),
offset_of_last(offset_of_last_)
{
const concurrent_queue_rep& rep = *queue.my_rep;
for( size_t k=0; k<concurrent_queue_rep::n_queue; ++k )
array[k] = rep.array[k].head_page;
}
//! Set item to point to kth element. Return true if at end of queue or item is marked valid; false otherwise.
bool get_item( void*& item, size_t k ) {
if( k==my_queue.my_rep->tail_counter ) {
item = NULL;
return true;
} else {
concurrent_queue_base::page* p = array[concurrent_queue_rep::index(k)];
__TBB_ASSERT(p,NULL);
size_t i = modulo_power_of_two( k/concurrent_queue_rep::n_queue, my_queue.items_per_page );
item = static_cast<unsigned char*>(static_cast<void*>(p)) + offset_of_last + my_queue.item_size*i;
return (p->mask & uintptr_t(1)<<i)!=0;
}
}
};
//------------------------------------------------------------------------
// concurrent_queue_iterator_base
//------------------------------------------------------------------------
void concurrent_queue_iterator_base_v3::initialize( const concurrent_queue_base& queue, size_t offset_of_last ) {
my_rep = cache_aligned_allocator<concurrent_queue_iterator_rep>().allocate(1);
new( my_rep ) concurrent_queue_iterator_rep(queue,offset_of_last);
size_t k = my_rep->head_counter;
if( !my_rep->get_item(my_item, k) ) advance();
}
concurrent_queue_iterator_base_v3::concurrent_queue_iterator_base_v3( const concurrent_queue_base& queue ) {
initialize(queue,0);
}
concurrent_queue_iterator_base_v3::concurrent_queue_iterator_base_v3( const concurrent_queue_base& queue, size_t offset_of_last ) {
initialize(queue,offset_of_last);
}
void concurrent_queue_iterator_base_v3::assign( const concurrent_queue_iterator_base& other ) {
if( my_rep!=other.my_rep ) {
if( my_rep ) {
cache_aligned_allocator<concurrent_queue_iterator_rep>().deallocate(my_rep, 1);
my_rep = NULL;
}
if( other.my_rep ) {
my_rep = cache_aligned_allocator<concurrent_queue_iterator_rep>().allocate(1);
new( my_rep ) concurrent_queue_iterator_rep( *other.my_rep );
}
}
my_item = other.my_item;
}
void concurrent_queue_iterator_base_v3::advance() {
__TBB_ASSERT( my_item, "attempt to increment iterator past end of queue" );
size_t k = my_rep->head_counter;
const concurrent_queue_base& queue = my_rep->my_queue;
#if TBB_USE_ASSERT
void* tmp;
my_rep->get_item(tmp,k);
__TBB_ASSERT( my_item==tmp, NULL );
#endif /* TBB_USE_ASSERT */
size_t i = modulo_power_of_two( k/concurrent_queue_rep::n_queue, queue.items_per_page );
if( i==queue.items_per_page-1 ) {
concurrent_queue_base::page*& root = my_rep->array[concurrent_queue_rep::index(k)];
root = root->next;
}
// advance k
my_rep->head_counter = ++k;
if( !my_rep->get_item(my_item, k) ) advance();
}
concurrent_queue_iterator_base_v3::~concurrent_queue_iterator_base_v3() {
//delete my_rep;
cache_aligned_allocator<concurrent_queue_iterator_rep>().deallocate(my_rep, 1);
my_rep = NULL;
}
} // namespace internal
} // namespace tbb

View File

@@ -0,0 +1,462 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_concurrent_queue_H
#define __TBB_concurrent_queue_H
#include "internal/_concurrent_queue_impl.h"
namespace tbb {
namespace strict_ppl {
//! A high-performance thread-safe non-blocking concurrent queue.
/** Multiple threads may each push and pop concurrently.
Assignment construction is not allowed.
@ingroup containers */
template<typename T, typename A = cache_aligned_allocator<T> >
class concurrent_queue: public internal::concurrent_queue_base_v3<T> {
template<typename Container, typename Value> friend class internal::concurrent_queue_iterator;
//! Allocator type
typedef typename A::template rebind<char>::other page_allocator_type;
page_allocator_type my_allocator;
//! Allocates a block of size n (bytes)
/*override*/ virtual void *allocate_block( size_t n ) {
void *b = reinterpret_cast<void*>(my_allocator.allocate( n ));
if( !b )
internal::throw_exception(internal::eid_bad_alloc);
return b;
}
//! Deallocates block created by allocate_block.
/*override*/ virtual void deallocate_block( void *b, size_t n ) {
my_allocator.deallocate( reinterpret_cast<char*>(b), n );
}
static void copy_construct_item(T* location, const void* src){
new (location) T(*static_cast<const T*>(src));
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
static void move_construct_item(T* location, const void* src) {
new (location) T( std::move(*static_cast<T*>(const_cast<void*>(src))) );
}
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
public:
//! Element type in the queue.
typedef T value_type;
//! Reference type
typedef T& reference;
//! Const reference type
typedef const T& const_reference;
//! Integral type for representing size of the queue.
typedef size_t size_type;
//! Difference type for iterator
typedef ptrdiff_t difference_type;
//! Allocator type
typedef A allocator_type;
//! Construct empty queue
explicit concurrent_queue(const allocator_type& a = allocator_type()) :
my_allocator( a )
{
}
//! [begin,end) constructor
template<typename InputIterator>
concurrent_queue( InputIterator begin, InputIterator end, const allocator_type& a = allocator_type()) :
my_allocator( a )
{
for( ; begin != end; ++begin )
this->push(*begin);
}
//! Copy constructor
concurrent_queue( const concurrent_queue& src, const allocator_type& a = allocator_type()) :
internal::concurrent_queue_base_v3<T>(), my_allocator( a )
{
this->assign( src, copy_construct_item );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Move constructors
concurrent_queue( concurrent_queue&& src ) :
internal::concurrent_queue_base_v3<T>(), my_allocator( std::move(src.my_allocator) )
{
this->internal_swap( src );
}
concurrent_queue( concurrent_queue&& src, const allocator_type& a ) :
internal::concurrent_queue_base_v3<T>(), my_allocator( a )
{
// checking that memory allocated by one instance of allocator can be deallocated
// with another
if( my_allocator == src.my_allocator) {
this->internal_swap( src );
} else {
// allocators are different => performing per-element move
this->assign( src, move_construct_item );
src.clear();
}
}
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! Destroy queue
~concurrent_queue();
//! Enqueue an item at tail of queue.
void push( const T& source ) {
this->internal_push( &source, copy_construct_item );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
void push( T&& source ) {
this->internal_push( &source, move_construct_item );
}
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
template<typename... Arguments>
void emplace( Arguments&&... args ) {
push( T(std::forward<Arguments>( args )...) );
}
#endif //__TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! Attempt to dequeue an item from head of queue.
/** Does not wait for item to become available.
Returns true if successful; false otherwise. */
bool try_pop( T& result ) {
return this->internal_try_pop( &result );
}
//! Return the number of items in the queue; thread unsafe
size_type unsafe_size() const {return this->internal_size();}
//! Equivalent to size()==0.
bool empty() const {return this->internal_empty();}
//! Clear the queue. not thread-safe.
void clear() ;
//! Return allocator object
allocator_type get_allocator() const { return this->my_allocator; }
typedef internal::concurrent_queue_iterator<concurrent_queue,T> iterator;
typedef internal::concurrent_queue_iterator<concurrent_queue,const T> const_iterator;
//------------------------------------------------------------------------
// The iterators are intended only for debugging. They are slow and not thread safe.
//------------------------------------------------------------------------
iterator unsafe_begin() {return iterator(*this);}
iterator unsafe_end() {return iterator();}
const_iterator unsafe_begin() const {return const_iterator(*this);}
const_iterator unsafe_end() const {return const_iterator();}
} ;
template<typename T, class A>
concurrent_queue<T,A>::~concurrent_queue() {
clear();
this->internal_finish_clear();
}
template<typename T, class A>
void concurrent_queue<T,A>::clear() {
while( !empty() ) {
T value;
this->internal_try_pop(&value);
}
}
} // namespace strict_ppl
//! A high-performance thread-safe blocking concurrent bounded queue.
/** This is the pre-PPL TBB concurrent queue which supports boundedness and blocking semantics.
Note that method names agree with the PPL-style concurrent queue.
Multiple threads may each push and pop concurrently.
Assignment construction is not allowed.
@ingroup containers */
template<typename T, class A = cache_aligned_allocator<T> >
class concurrent_bounded_queue: public internal::concurrent_queue_base_v8 {
template<typename Container, typename Value> friend class internal::concurrent_queue_iterator;
//! Allocator type
typedef typename A::template rebind<char>::other page_allocator_type;
page_allocator_type my_allocator;
typedef typename concurrent_queue_base_v3::padded_page<T> padded_page;
typedef typename concurrent_queue_base_v3::copy_specifics copy_specifics;
//! Class used to ensure exception-safety of method "pop"
class destroyer: internal::no_copy {
T& my_value;
public:
destroyer( T& value ) : my_value(value) {}
~destroyer() {my_value.~T();}
};
T& get_ref( page& p, size_t index ) {
__TBB_ASSERT( index<items_per_page, NULL );
return (&static_cast<padded_page*>(static_cast<void*>(&p))->last)[index];
}
/*override*/ virtual void copy_item( page& dst, size_t index, const void* src ) {
new( &get_ref(dst,index) ) T(*static_cast<const T*>(src));
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
/*override*/ virtual void move_item( page& dst, size_t index, const void* src ) {
new( &get_ref(dst,index) ) T( std::move(*static_cast<T*>(const_cast<void*>(src))) );
}
#else
/*override*/ virtual void move_item( page&, size_t, const void* ) {
__TBB_ASSERT( false, "Unreachable code" );
}
#endif
/*override*/ virtual void copy_page_item( page& dst, size_t dindex, const page& src, size_t sindex ) {
new( &get_ref(dst,dindex) ) T( get_ref( const_cast<page&>(src), sindex ) );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
/*override*/ virtual void move_page_item( page& dst, size_t dindex, const page& src, size_t sindex ) {
new( &get_ref(dst,dindex) ) T( std::move(get_ref( const_cast<page&>(src), sindex )) );
}
#else
/*override*/ virtual void move_page_item( page&, size_t, const page&, size_t ) {
__TBB_ASSERT( false, "Unreachable code" );
}
#endif
/*override*/ virtual void assign_and_destroy_item( void* dst, page& src, size_t index ) {
T& from = get_ref(src,index);
destroyer d(from);
*static_cast<T*>(dst) = tbb::internal::move( from );
}
/*override*/ virtual page *allocate_page() {
size_t n = sizeof(padded_page) + (items_per_page-1)*sizeof(T);
page *p = reinterpret_cast<page*>(my_allocator.allocate( n ));
if( !p )
internal::throw_exception(internal::eid_bad_alloc);
return p;
}
/*override*/ virtual void deallocate_page( page *p ) {
size_t n = sizeof(padded_page) + (items_per_page-1)*sizeof(T);
my_allocator.deallocate( reinterpret_cast<char*>(p), n );
}
public:
//! Element type in the queue.
typedef T value_type;
//! Allocator type
typedef A allocator_type;
//! Reference type
typedef T& reference;
//! Const reference type
typedef const T& const_reference;
//! Integral type for representing size of the queue.
/** Note that the size_type is a signed integral type.
This is because the size can be negative if there are pending pops without corresponding pushes. */
typedef std::ptrdiff_t size_type;
//! Difference type for iterator
typedef std::ptrdiff_t difference_type;
//! Construct empty queue
explicit concurrent_bounded_queue(const allocator_type& a = allocator_type()) :
concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
{
}
//! Copy constructor
concurrent_bounded_queue( const concurrent_bounded_queue& src, const allocator_type& a = allocator_type())
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
{
assign( src );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Move constructors
concurrent_bounded_queue( concurrent_bounded_queue&& src )
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( std::move(src.my_allocator) )
{
internal_swap( src );
}
concurrent_bounded_queue( concurrent_bounded_queue&& src, const allocator_type& a )
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
{
// checking that memory allocated by one instance of allocator can be deallocated
// with another
if( my_allocator == src.my_allocator) {
this->internal_swap( src );
} else {
// allocators are different => performing per-element move
this->move_content( src );
src.clear();
}
}
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! [begin,end) constructor
template<typename InputIterator>
concurrent_bounded_queue( InputIterator begin, InputIterator end,
const allocator_type& a = allocator_type())
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
{
for( ; begin != end; ++begin )
internal_push_if_not_full(&*begin);
}
//! Destroy queue
~concurrent_bounded_queue();
//! Enqueue an item at tail of queue.
void push( const T& source ) {
internal_push( &source );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Move an item at tail of queue.
void push( T&& source ) {
internal_push_move( &source );
}
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
template<typename... Arguments>
void emplace( Arguments&&... args ) {
push( T(std::forward<Arguments>( args )...) );
}
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! Dequeue item from head of queue.
/** Block until an item becomes available, and then dequeue it. */
void pop( T& destination ) {
internal_pop( &destination );
}
#if TBB_USE_EXCEPTIONS
//! Abort all pending queue operations
void abort() {
internal_abort();
}
#endif
//! Enqueue an item at tail of queue if queue is not already full.
/** Does not wait for queue to become not full.
Returns true if item is pushed; false if queue was already full. */
bool try_push( const T& source ) {
return internal_push_if_not_full( &source );
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
//! Move an item at tail of queue if queue is not already full.
/** Does not wait for queue to become not full.
Returns true if item is pushed; false if queue was already full. */
bool try_push( T&& source ) {
return internal_push_move_if_not_full( &source );
}
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
template<typename... Arguments>
bool try_emplace( Arguments&&... args ) {
return try_push( T(std::forward<Arguments>( args )...) );
}
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
//! Attempt to dequeue an item from head of queue.
/** Does not wait for item to become available.
Returns true if successful; false otherwise. */
bool try_pop( T& destination ) {
return internal_pop_if_present( &destination );
}
//! Return number of pushes minus number of pops.
/** Note that the result can be negative if there are pops waiting for the
corresponding pushes. The result can also exceed capacity() if there
are push operations in flight. */
size_type size() const {return internal_size();}
//! Equivalent to size()<=0.
bool empty() const {return internal_empty();}
//! Maximum number of allowed elements
size_type capacity() const {
return my_capacity;
}
//! Set the capacity
/** Setting the capacity to 0 causes subsequent try_push operations to always fail,
and subsequent push operations to block forever. */
void set_capacity( size_type new_capacity ) {
internal_set_capacity( new_capacity, sizeof(T) );
}
//! return allocator object
allocator_type get_allocator() const { return this->my_allocator; }
//! clear the queue. not thread-safe.
void clear() ;
typedef internal::concurrent_queue_iterator<concurrent_bounded_queue,T> iterator;
typedef internal::concurrent_queue_iterator<concurrent_bounded_queue,const T> const_iterator;
//------------------------------------------------------------------------
// The iterators are intended only for debugging. They are slow and not thread safe.
//------------------------------------------------------------------------
iterator unsafe_begin() {return iterator(*this);}
iterator unsafe_end() {return iterator();}
const_iterator unsafe_begin() const {return const_iterator(*this);}
const_iterator unsafe_end() const {return const_iterator();}
};
template<typename T, class A>
concurrent_bounded_queue<T,A>::~concurrent_bounded_queue() {
clear();
internal_finish_clear();
}
template<typename T, class A>
void concurrent_bounded_queue<T,A>::clear() {
while( !empty() ) {
T value;
internal_pop_if_present(&value);
}
}
using strict_ppl::concurrent_queue;
} // namespace tbb
#endif /* __TBB_concurrent_queue_H */

View File

@@ -0,0 +1,326 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
/* Container implementations in this header are based on PPL implementations
provided by Microsoft. */
#ifndef __TBB_concurrent_unordered_map_H
#define __TBB_concurrent_unordered_map_H
#include "internal/_concurrent_unordered_impl.h"
namespace tbb
{
namespace interface5 {
// Template class for hash map traits
template<typename Key, typename T, typename Hash_compare, typename Allocator, bool Allow_multimapping>
class concurrent_unordered_map_traits
{
protected:
typedef std::pair<const Key, T> value_type;
typedef Key key_type;
typedef Hash_compare hash_compare;
typedef typename Allocator::template rebind<value_type>::other allocator_type;
enum { allow_multimapping = Allow_multimapping };
concurrent_unordered_map_traits() : my_hash_compare() {}
concurrent_unordered_map_traits(const hash_compare& hc) : my_hash_compare(hc) {}
class value_compare : public std::binary_function<value_type, value_type, bool>
{
friend class concurrent_unordered_map_traits<Key, T, Hash_compare, Allocator, Allow_multimapping>;
public:
bool operator()(const value_type& left, const value_type& right) const
{
return (my_hash_compare(left.first, right.first));
}
value_compare(const hash_compare& comparator) : my_hash_compare(comparator) {}
protected:
hash_compare my_hash_compare; // the comparator predicate for keys
};
template<class Type1, class Type2>
static const Key& get_key(const std::pair<Type1, Type2>& value) {
return (value.first);
}
hash_compare my_hash_compare; // the comparator predicate for keys
};
template <typename Key, typename T, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
typename Allocator = tbb::tbb_allocator<std::pair<const Key, T> > >
class concurrent_unordered_map :
public internal::concurrent_unordered_base< concurrent_unordered_map_traits<Key, T,
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> >
{
// Base type definitions
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
typedef concurrent_unordered_map_traits<Key, T, hash_compare, Allocator, false> traits_type;
typedef internal::concurrent_unordered_base< traits_type > base_type;
#if __TBB_EXTRA_DEBUG
public:
#endif
using traits_type::allow_multimapping;
public:
using base_type::end;
using base_type::find;
using base_type::insert;
// Type definitions
typedef Key key_type;
typedef typename base_type::value_type value_type;
typedef T mapped_type;
typedef Hasher hasher;
typedef Key_equality key_equal;
typedef hash_compare key_compare;
typedef typename base_type::allocator_type allocator_type;
typedef typename base_type::pointer pointer;
typedef typename base_type::const_pointer const_pointer;
typedef typename base_type::reference reference;
typedef typename base_type::const_reference const_reference;
typedef typename base_type::size_type size_type;
typedef typename base_type::difference_type difference_type;
typedef typename base_type::iterator iterator;
typedef typename base_type::const_iterator const_iterator;
typedef typename base_type::iterator local_iterator;
typedef typename base_type::const_iterator const_local_iterator;
// Construction/destruction/copying
explicit concurrent_unordered_map(size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
}
concurrent_unordered_map(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
{
}
template <typename Iterator>
concurrent_unordered_map(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
insert(first, last);
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Constructor from initializer_list
concurrent_unordered_map(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
this->insert(il.begin(),il.end());
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_map(const concurrent_unordered_map& table)
: base_type(table)
{
}
concurrent_unordered_map& operator=(const concurrent_unordered_map& table)
{
return static_cast<concurrent_unordered_map&>(base_type::operator=(table));
}
concurrent_unordered_map(concurrent_unordered_map&& table)
: base_type(std::move(table))
{
}
concurrent_unordered_map& operator=(concurrent_unordered_map&& table)
{
return static_cast<concurrent_unordered_map&>(base_type::operator=(std::move(table)));
}
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_map(const concurrent_unordered_map& table, const Allocator& a)
: base_type(table, a)
{
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
concurrent_unordered_map(concurrent_unordered_map&& table, const Allocator& a) : base_type(std::move(table), a)
{
}
#endif
// Observers
mapped_type& operator[](const key_type& key)
{
iterator where = find(key);
if (where == end())
{
where = insert(std::pair<key_type, mapped_type>(key, mapped_type())).first;
}
return ((*where).second);
}
mapped_type& at(const key_type& key)
{
iterator where = find(key);
if (where == end())
{
tbb::internal::throw_exception(tbb::internal::eid_invalid_key);
}
return ((*where).second);
}
const mapped_type& at(const key_type& key) const
{
const_iterator where = find(key);
if (where == end())
{
tbb::internal::throw_exception(tbb::internal::eid_invalid_key);
}
return ((*where).second);
}
};
template < typename Key, typename T, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
typename Allocator = tbb::tbb_allocator<std::pair<const Key, T> > >
class concurrent_unordered_multimap :
public internal::concurrent_unordered_base< concurrent_unordered_map_traits< Key, T,
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, true> >
{
// Base type definitions
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
typedef concurrent_unordered_map_traits<Key, T, hash_compare, Allocator, true> traits_type;
typedef internal::concurrent_unordered_base<traits_type> base_type;
#if __TBB_EXTRA_DEBUG
public:
#endif
using traits_type::allow_multimapping;
public:
using base_type::insert;
// Type definitions
typedef Key key_type;
typedef typename base_type::value_type value_type;
typedef T mapped_type;
typedef Hasher hasher;
typedef Key_equality key_equal;
typedef hash_compare key_compare;
typedef typename base_type::allocator_type allocator_type;
typedef typename base_type::pointer pointer;
typedef typename base_type::const_pointer const_pointer;
typedef typename base_type::reference reference;
typedef typename base_type::const_reference const_reference;
typedef typename base_type::size_type size_type;
typedef typename base_type::difference_type difference_type;
typedef typename base_type::iterator iterator;
typedef typename base_type::const_iterator const_iterator;
typedef typename base_type::iterator local_iterator;
typedef typename base_type::const_iterator const_local_iterator;
// Construction/destruction/copying
explicit concurrent_unordered_multimap(size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
}
concurrent_unordered_multimap(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
{
}
template <typename Iterator>
concurrent_unordered_multimap(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets,key_compare(_Hasher,_Key_equality), a)
{
insert(first, last);
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Constructor from initializer_list
concurrent_unordered_multimap(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
this->insert(il.begin(),il.end());
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_multimap(const concurrent_unordered_multimap& table)
: base_type(table)
{
}
concurrent_unordered_multimap& operator=(const concurrent_unordered_multimap& table)
{
return static_cast<concurrent_unordered_multimap&>(base_type::operator=(table));
}
concurrent_unordered_multimap(concurrent_unordered_multimap&& table)
: base_type(std::move(table))
{
}
concurrent_unordered_multimap& operator=(concurrent_unordered_multimap&& table)
{
return static_cast<concurrent_unordered_multimap&>(base_type::operator=(std::move(table)));
}
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_multimap(const concurrent_unordered_multimap& table, const Allocator& a)
: base_type(table, a)
{
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
concurrent_unordered_multimap(concurrent_unordered_multimap&& table, const Allocator& a) : base_type(std::move(table), a)
{
}
#endif
};
} // namespace interface5
using interface5::concurrent_unordered_map;
using interface5::concurrent_unordered_multimap;
} // namespace tbb
#endif// __TBB_concurrent_unordered_map_H

View File

@@ -0,0 +1,269 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
/* Container implementations in this header are based on PPL implementations
provided by Microsoft. */
#ifndef __TBB_concurrent_unordered_set_H
#define __TBB_concurrent_unordered_set_H
#include "internal/_concurrent_unordered_impl.h"
namespace tbb
{
namespace interface5 {
// Template class for hash set traits
template<typename Key, typename Hash_compare, typename Allocator, bool Allow_multimapping>
class concurrent_unordered_set_traits
{
protected:
typedef Key value_type;
typedef Key key_type;
typedef Hash_compare hash_compare;
typedef typename Allocator::template rebind<value_type>::other allocator_type;
enum { allow_multimapping = Allow_multimapping };
concurrent_unordered_set_traits() : my_hash_compare() {}
concurrent_unordered_set_traits(const hash_compare& hc) : my_hash_compare(hc) {}
typedef hash_compare value_compare;
static const Key& get_key(const value_type& value) {
return value;
}
hash_compare my_hash_compare; // the comparator predicate for keys
};
template <typename Key, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>, typename Allocator = tbb::tbb_allocator<Key> >
class concurrent_unordered_set : public internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key, internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> >
{
// Base type definitions
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
typedef internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key, hash_compare, Allocator, false> > base_type;
typedef concurrent_unordered_set_traits<Key, internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> traits_type;
#if __TBB_EXTRA_DEBUG
public:
#endif
using traits_type::allow_multimapping;
public:
using base_type::insert;
// Type definitions
typedef Key key_type;
typedef typename base_type::value_type value_type;
typedef Key mapped_type;
typedef Hasher hasher;
typedef Key_equality key_equal;
typedef hash_compare key_compare;
typedef typename base_type::allocator_type allocator_type;
typedef typename base_type::pointer pointer;
typedef typename base_type::const_pointer const_pointer;
typedef typename base_type::reference reference;
typedef typename base_type::const_reference const_reference;
typedef typename base_type::size_type size_type;
typedef typename base_type::difference_type difference_type;
typedef typename base_type::iterator iterator;
typedef typename base_type::const_iterator const_iterator;
typedef typename base_type::iterator local_iterator;
typedef typename base_type::const_iterator const_local_iterator;
// Construction/destruction/copying
explicit concurrent_unordered_set(size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
{
}
concurrent_unordered_set(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
{
}
template <typename Iterator>
concurrent_unordered_set(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
{
insert(first, last);
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Constructor from initializer_list
concurrent_unordered_set(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
{
this->insert(il.begin(),il.end());
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_set(const concurrent_unordered_set& table)
: base_type(table)
{
}
concurrent_unordered_set& operator=(const concurrent_unordered_set& table)
{
return static_cast<concurrent_unordered_set&>(base_type::operator=(table));
}
concurrent_unordered_set(concurrent_unordered_set&& table)
: base_type(std::move(table))
{
}
concurrent_unordered_set& operator=(concurrent_unordered_set&& table)
{
return static_cast<concurrent_unordered_set&>(base_type::operator=(std::move(table)));
}
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_set(const concurrent_unordered_set& table, const Allocator& a)
: base_type(table, a)
{
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
concurrent_unordered_set(concurrent_unordered_set&& table, const Allocator& a)
: base_type(std::move(table), a)
{
}
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
};
template <typename Key, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
typename Allocator = tbb::tbb_allocator<Key> >
class concurrent_unordered_multiset :
public internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key,
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, true> >
{
// Base type definitions
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
typedef concurrent_unordered_set_traits<Key, hash_compare, Allocator, true> traits_type;
typedef internal::concurrent_unordered_base< traits_type > base_type;
#if __TBB_EXTRA_DEBUG
public:
#endif
using traits_type::allow_multimapping;
public:
using base_type::insert;
// Type definitions
typedef Key key_type;
typedef typename base_type::value_type value_type;
typedef Key mapped_type;
typedef Hasher hasher;
typedef Key_equality key_equal;
typedef hash_compare key_compare;
typedef typename base_type::allocator_type allocator_type;
typedef typename base_type::pointer pointer;
typedef typename base_type::const_pointer const_pointer;
typedef typename base_type::reference reference;
typedef typename base_type::const_reference const_reference;
typedef typename base_type::size_type size_type;
typedef typename base_type::difference_type difference_type;
typedef typename base_type::iterator iterator;
typedef typename base_type::const_iterator const_iterator;
typedef typename base_type::iterator local_iterator;
typedef typename base_type::const_iterator const_local_iterator;
// Construction/destruction/copying
explicit concurrent_unordered_multiset(size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
}
concurrent_unordered_multiset(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
{
}
template <typename Iterator>
concurrent_unordered_multiset(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
{
insert(first, last);
}
#if __TBB_INITIALIZER_LISTS_PRESENT
//! Constructor from initializer_list
concurrent_unordered_multiset(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
{
this->insert(il.begin(),il.end());
}
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_multiset(const concurrent_unordered_multiset& table)
: base_type(table)
{
}
concurrent_unordered_multiset& operator=(const concurrent_unordered_multiset& table)
{
return static_cast<concurrent_unordered_multiset&>(base_type::operator=(table));
}
concurrent_unordered_multiset(concurrent_unordered_multiset&& table)
: base_type(std::move(table))
{
}
concurrent_unordered_multiset& operator=(concurrent_unordered_multiset&& table)
{
return static_cast<concurrent_unordered_multiset&>(base_type::operator=(std::move(table)));
}
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
concurrent_unordered_multiset(const concurrent_unordered_multiset& table, const Allocator& a)
: base_type(table, a)
{
}
#if __TBB_CPP11_RVALUE_REF_PRESENT
concurrent_unordered_multiset(concurrent_unordered_multiset&& table, const Allocator& a)
: base_type(std::move(table), a)
{
}
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
};
} // namespace interface5
using interface5::concurrent_unordered_set;
using interface5::concurrent_unordered_multiset;
} // namespace tbb
#endif// __TBB_concurrent_unordered_set_H

View File

@@ -0,0 +1,631 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if (_MSC_VER)
//MSVC 10 "deprecated" application of some std:: algorithms to raw pointers as not safe.
//The reason is that destination is not checked against bounds/having enough place.
#define _SCL_SECURE_NO_WARNINGS
#endif
#include "tbb/concurrent_vector.h"
#include "tbb/cache_aligned_allocator.h"
#include "tbb/tbb_exception.h"
#include "tbb_misc.h"
#include "itt_notify.h"
#if !TBB_USE_EXCEPTIONS && _MSC_VER
// Suppress "C++ exception handler used, but unwind semantics are not enabled" warning in STL headers
#pragma warning (push)
#pragma warning (disable: 4530)
#endif
#include <cstring>
#include <memory> //for uninitialized_fill_n
#if !TBB_USE_EXCEPTIONS && _MSC_VER
#pragma warning (pop)
#endif
#if defined(_MSC_VER) && defined(_Wp64)
// Workaround for overzealous compiler warnings in /Wp64 mode
#pragma warning (disable: 4267)
#endif
using namespace std;
namespace tbb {
namespace internal {
class concurrent_vector_base_v3::helper :no_assign {
public:
//! memory page size
static const size_type page_size = 4096;
inline static bool incompact_predicate(size_type size) { // assert size != 0, see source/test/test_vector_layout.cpp
return size < page_size || ((size-1)%page_size < page_size/2 && size < page_size * 128); // for more details
}
inline static size_type find_segment_end(const concurrent_vector_base_v3 &v) {
segment_t *s = v.my_segment;
segment_index_t u = s==v.my_storage? pointers_per_short_table : pointers_per_long_table;
segment_index_t k = 0;
while( k < u && (s[k].load<relaxed>()==segment_allocated() ))
++k;
return k;
}
// TODO: optimize accesses to my_first_block
//! assign first segment size. k - is index of last segment to be allocated, not a count of segments
inline static void assign_first_segment_if_necessary(concurrent_vector_base_v3 &v, segment_index_t k) {
if( !v.my_first_block ) {
/* There was a suggestion to set first segment according to incompact_predicate:
while( k && !helper::incompact_predicate(segment_size( k ) * element_size) )
--k; // while previous vector size is compact, decrement
// reasons to not do it:
// * constructor(n) is not ready to accept fragmented segments
// * backward compatibility due to that constructor
// * current version gives additional guarantee and faster init.
// * two calls to reserve() will give the same effect.
*/
v.my_first_block.compare_and_swap(k+1, 0); // store number of segments
}
}
inline static void *allocate_segment(concurrent_vector_base_v3 &v, size_type n) {
void *ptr = v.vector_allocator_ptr(v, n);
if(!ptr) throw_exception(eid_bad_alloc); // check for bad allocation, throw exception
return ptr;
}
//! Publish segment so other threads can see it.
template<typename argument_type>
inline static void publish_segment( segment_t& s, argument_type rhs ) {
// see also itt_store_pointer_with_release_v3()
ITT_NOTIFY( sync_releasing, &s );
s.store<release>(rhs);
}
static size_type enable_segment(concurrent_vector_base_v3 &v, size_type k, size_type element_size, bool mark_as_not_used_on_failure = false);
// TODO: rename as get_segments_table() and return segment pointer
inline static void extend_table_if_necessary(concurrent_vector_base_v3 &v, size_type k, size_type start ) {
if(k >= pointers_per_short_table && v.my_segment == v.my_storage)
extend_segment_table(v, start );
}
static void extend_segment_table(concurrent_vector_base_v3 &v, size_type start);
struct segment_not_used_predicate: no_assign {
segment_t &s;
segment_not_used_predicate(segment_t &segment) : s(segment) {}
bool operator()() const { return s.load<relaxed>() == segment_not_used ();}
};
inline static segment_t& acquire_segment(concurrent_vector_base_v3 &v, size_type index, size_type element_size, bool owner) {
segment_t &s = v.my_segment[index]; // TODO: pass v.my_segment as argument
if( s.load<acquire>() == segment_not_used() ) { // do not check for segment_allocation_failed state
if( owner ) {
enable_segment( v, index, element_size );
} else {
ITT_NOTIFY(sync_prepare, &s);
spin_wait_while(segment_not_used_predicate(s));
ITT_NOTIFY(sync_acquired, &s);
}
} else {
ITT_NOTIFY(sync_acquired, &s);
}
if(s.load<relaxed>() != segment_allocated())
throw_exception(eid_bad_last_alloc); // throw custom exception, because it's hard to recover correctly after segment_allocation_failed state
return s;
}
///// non-static fields of helper for exception-safe iteration across segments
segment_t *table;// TODO: review all segment_index_t as just short type
size_type first_block, k, sz, start, finish, element_size;
helper(segment_t *segments, size_type fb, size_type esize, size_type index, size_type s, size_type f) throw()
: table(segments), first_block(fb), k(index), sz(0), start(s), finish(f), element_size(esize) {}
inline void first_segment() throw() {
__TBB_ASSERT( start <= finish, NULL );
__TBB_ASSERT( first_block || !finish, NULL );
if( k < first_block ) k = 0; // process solid segment at a time
size_type base = segment_base( k );
__TBB_ASSERT( base <= start, NULL );
finish -= base; start -= base; // rebase as offsets from segment k
sz = k ? base : segment_size( first_block ); // sz==base for k>0
}
inline void next_segment() throw() {
finish -= sz; start = 0; // offsets from next segment
if( !k ) k = first_block;
else { ++k; sz = segment_size( k ); }
}
template<typename F>
inline size_type apply(const F &func) {
first_segment();
while( sz < finish ) { // work for more than one segment
//TODO: remove extra load() of table[k] inside func
func( table[k], table[k].load<relaxed>().pointer<char>() + element_size*start, sz - start );
next_segment();
}
func( table[k], table[k].load<relaxed>().pointer<char>() + element_size*start, finish - start );
return k;
}
inline segment_value_t get_segment_value(size_type index, bool wait) {
segment_t &s = table[index];
if( wait && (s.load<acquire>() == segment_not_used()) ) {
ITT_NOTIFY(sync_prepare, &s);
spin_wait_while(segment_not_used_predicate(s));
ITT_NOTIFY(sync_acquired, &s);
}
return s.load<relaxed>();
}
~helper() {
if( sz >= finish ) return; // the work is done correctly
cleanup();
}
//! Out of line code to assists destructor in infrequent cases.
void cleanup();
/// TODO: turn into lambda functions when available
struct init_body {
internal_array_op2 func;
const void *arg;
init_body(internal_array_op2 init, const void *src) : func(init), arg(src) {}
void operator()(segment_t &, void *begin, size_type n) const {
func( begin, arg, n );
}
};
struct safe_init_body {
internal_array_op2 func;
const void *arg;
safe_init_body(internal_array_op2 init, const void *src) : func(init), arg(src) {}
void operator()(segment_t &s, void *begin, size_type n) const {
if(s.load<relaxed>() != segment_allocated())
throw_exception(eid_bad_last_alloc); // throw custom exception
func( begin, arg, n );
}
};
struct destroy_body {
internal_array_op1 func;
destroy_body(internal_array_op1 destroy) : func(destroy) {}
void operator()(segment_t &s, void *begin, size_type n) const {
if(s.load<relaxed>() == segment_allocated())
func( begin, n );
}
};
};
void concurrent_vector_base_v3::helper::extend_segment_table(concurrent_vector_base_v3 &v, concurrent_vector_base_v3::size_type start) {
if( start > segment_size(pointers_per_short_table) ) start = segment_size(pointers_per_short_table);
// If other threads are trying to set pointers in the short segment, wait for them to finish their
// assignments before we copy the short segment to the long segment. Note: grow_to_at_least depends on it
for( segment_index_t i = 0; segment_base(i) < start && v.my_segment == v.my_storage; i++ ){
if(v.my_storage[i].load<relaxed>() == segment_not_used()) {
ITT_NOTIFY(sync_prepare, &v.my_storage[i]);
atomic_backoff backoff(true);
while( v.my_segment == v.my_storage && (v.my_storage[i].load<relaxed>() == segment_not_used()) )
backoff.pause();
ITT_NOTIFY(sync_acquired, &v.my_storage[i]);
}
}
if( v.my_segment != v.my_storage ) return;
segment_t* new_segment_table = (segment_t*)NFS_Allocate( pointers_per_long_table, sizeof(segment_t), NULL );
__TBB_ASSERT(new_segment_table, "NFS_Allocate should throws exception if it cannot allocate the requested storage, and not returns zero pointer" );
std::uninitialized_fill_n(new_segment_table,size_t(pointers_per_long_table),segment_t()); //init newly allocated table
//TODO: replace with static assert
__TBB_STATIC_ASSERT(pointers_per_long_table >= pointers_per_short_table, "size of the big table should be not lesser than of the small one, as we copy values to it" );
std::copy(v.my_storage, v.my_storage+pointers_per_short_table, new_segment_table);//copy values from old table, here operator= of segment_t is used
if( v.my_segment.compare_and_swap( new_segment_table, v.my_storage ) != v.my_storage )
NFS_Free( new_segment_table );
// else TODO: add ITT_NOTIFY signals for v.my_segment?
}
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::helper::enable_segment(concurrent_vector_base_v3 &v, concurrent_vector_base_v3::size_type k, concurrent_vector_base_v3::size_type element_size,
bool mark_as_not_used_on_failure ) {
struct segment_scope_guard : no_copy{
segment_t* my_segment_ptr;
bool my_mark_as_not_used;
segment_scope_guard(segment_t& segment, bool mark_as_not_used) : my_segment_ptr(&segment), my_mark_as_not_used(mark_as_not_used){}
void dismiss(){ my_segment_ptr = 0;}
~segment_scope_guard(){
if (my_segment_ptr){
if (!my_mark_as_not_used){
publish_segment(*my_segment_ptr, segment_allocation_failed());
}else{
publish_segment(*my_segment_ptr, segment_not_used());
}
}
}
};
segment_t* s = v.my_segment; // TODO: optimize out as argument? Optimize accesses to my_first_block
__TBB_ASSERT(s[k].load<relaxed>() != segment_allocated(), "concurrent operation during growth?");
size_type size_of_enabled_segment = segment_size(k);
size_type size_to_allocate = size_of_enabled_segment;
if( !k ) {
assign_first_segment_if_necessary(v, default_initial_segments-1);
size_of_enabled_segment = 2 ;
size_to_allocate = segment_size(v.my_first_block);
} else {
spin_wait_while_eq( v.my_first_block, segment_index_t(0) );
}
if( k && (k < v.my_first_block)){ //no need to allocate anything
// s[0].array is changed only once ( 0 -> !0 ) and points to uninitialized memory
segment_value_t array0 = s[0].load<acquire>();
if(array0 == segment_not_used()){
// sync_prepare called only if there is a wait
ITT_NOTIFY(sync_prepare, &s[0]);
spin_wait_while( segment_not_used_predicate(s[0]));
array0 = s[0].load<acquire>();
}
ITT_NOTIFY(sync_acquired, &s[0]);
if(array0 != segment_allocated()) { // check for segment_allocation_failed state of initial segment
publish_segment(s[k], segment_allocation_failed()); // and assign segment_allocation_failed state here
throw_exception(eid_bad_last_alloc); // throw custom exception
}
publish_segment( s[k],
static_cast<void*>(array0.pointer<char>() + segment_base(k)*element_size )
);
} else {
segment_scope_guard k_segment_guard(s[k], mark_as_not_used_on_failure);
publish_segment(s[k], allocate_segment(v, size_to_allocate));
k_segment_guard.dismiss();
}
return size_of_enabled_segment;
}
void concurrent_vector_base_v3::helper::cleanup() {
if( !sz ) { // allocation failed, restore the table
segment_index_t k_start = k, k_end = segment_index_of(finish-1);
if( segment_base( k_start ) < start )
get_segment_value(k_start++, true); // wait
if( k_start < first_block ) {
segment_value_t segment0 = get_segment_value(0, start>0); // wait if necessary
if((segment0 != segment_not_used()) && !k_start ) ++k_start;
if(segment0 != segment_allocated())
for(; k_start < first_block && k_start <= k_end; ++k_start )
publish_segment(table[k_start], segment_allocation_failed());
else for(; k_start < first_block && k_start <= k_end; ++k_start )
publish_segment(table[k_start], static_cast<void*>(
(segment0.pointer<char>()) + segment_base(k_start)*element_size) );
}
for(; k_start <= k_end; ++k_start ) // not in first block
if(table[k_start].load<acquire>() == segment_not_used())
publish_segment(table[k_start], segment_allocation_failed());
// fill allocated items
first_segment();
goto recover;
}
while( sz <= finish ) { // there is still work for at least one segment
next_segment();
recover:
segment_value_t array = table[k].load<relaxed>();
if(array == segment_allocated())
std::memset( (array.pointer<char>()) + element_size*start, 0, ((sz<finish?sz:finish) - start)*element_size );
else __TBB_ASSERT( array == segment_allocation_failed(), NULL );
}
}
concurrent_vector_base_v3::~concurrent_vector_base_v3() {
segment_t* s = my_segment;
if( s != my_storage ) {
#if TBB_USE_ASSERT
//to please assert in segment_t destructor
std::fill_n(my_storage,size_t(pointers_per_short_table),segment_t());
#endif /* TBB_USE_ASSERT */
#if TBB_USE_DEBUG
for( segment_index_t i = 0; i < pointers_per_long_table; i++)
__TBB_ASSERT( my_segment[i].load<relaxed>() != segment_allocated(), "Segment should have been freed. Please recompile with new TBB before using exceptions.");
#endif
my_segment = my_storage;
NFS_Free( s );
}
}
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_capacity() const {
return segment_base( helper::find_segment_end(*this) );
}
void concurrent_vector_base_v3::internal_throw_exception(size_type t) const {
switch(t) {
case 0: throw_exception(eid_out_of_range);
case 1: throw_exception(eid_segment_range_error);
case 2: throw_exception(eid_index_range_error);
}
}
void concurrent_vector_base_v3::internal_reserve( size_type n, size_type element_size, size_type max_size ) {
if( n>max_size )
throw_exception(eid_reservation_length_error);
__TBB_ASSERT( n, NULL );
helper::assign_first_segment_if_necessary(*this, segment_index_of(n-1));
segment_index_t k = helper::find_segment_end(*this);
for( ; segment_base(k)<n; ++k ) {
helper::extend_table_if_necessary(*this, k, 0);
if(my_segment[k].load<relaxed>() != segment_allocated())
helper::enable_segment(*this, k, element_size, true ); //in case of failure mark segments as not used
}
}
//TODO: Looks like atomic loads can be done relaxed here, as the only place this method is called from
//is the constructor, which does not require synchronization (for more details see comment in the
// concurrent_vector_base constructor).
void concurrent_vector_base_v3::internal_copy( const concurrent_vector_base_v3& src, size_type element_size, internal_array_op2 copy ) {
size_type n = src.my_early_size;
__TBB_ASSERT( my_segment == my_storage, NULL);
if( n ) {
helper::assign_first_segment_if_necessary(*this, segment_index_of(n-1));
size_type b;
for( segment_index_t k=0; (b=segment_base(k))<n; ++k ) {
if( (src.my_segment.load<acquire>() == src.my_storage && k >= pointers_per_short_table)
|| (src.my_segment[k].load<relaxed>() != segment_allocated())) {
my_early_size = b; break;
}
helper::extend_table_if_necessary(*this, k, 0);
size_type m = helper::enable_segment(*this, k, element_size);
if( m > n-b ) m = n-b;
my_early_size = b+m;
copy( my_segment[k].load<relaxed>().pointer<void>(), src.my_segment[k].load<relaxed>().pointer<void>(), m );
}
}
}
void concurrent_vector_base_v3::internal_assign( const concurrent_vector_base_v3& src, size_type element_size, internal_array_op1 destroy, internal_array_op2 assign, internal_array_op2 copy ) {
size_type n = src.my_early_size;
while( my_early_size>n ) { // TODO: improve
segment_index_t k = segment_index_of( my_early_size-1 );
size_type b=segment_base(k);
size_type new_end = b>=n ? b : n;
__TBB_ASSERT( my_early_size>new_end, NULL );
if( my_segment[k].load<relaxed>() != segment_allocated()) // check vector was broken before
throw_exception(eid_bad_last_alloc); // throw custom exception
// destructors are supposed to not throw any exceptions
destroy( my_segment[k].load<relaxed>().pointer<char>() + element_size*(new_end-b), my_early_size-new_end );
my_early_size = new_end;
}
size_type dst_initialized_size = my_early_size;
my_early_size = n;
helper::assign_first_segment_if_necessary(*this, segment_index_of(n));
size_type b;
for( segment_index_t k=0; (b=segment_base(k))<n; ++k ) {
if( (src.my_segment.load<acquire>() == src.my_storage && k >= pointers_per_short_table)
|| src.my_segment[k].load<relaxed>() != segment_allocated() ) { // if source is damaged
my_early_size = b; break; // TODO: it may cause undestructed items
}
helper::extend_table_if_necessary(*this, k, 0);
if( my_segment[k].load<relaxed>() == segment_not_used())
helper::enable_segment(*this, k, element_size);
else if( my_segment[k].load<relaxed>() != segment_allocated() )
throw_exception(eid_bad_last_alloc); // throw custom exception
size_type m = k? segment_size(k) : 2;
if( m > n-b ) m = n-b;
size_type a = 0;
if( dst_initialized_size>b ) {
a = dst_initialized_size-b;
if( a>m ) a = m;
assign( my_segment[k].load<relaxed>().pointer<void>(), src.my_segment[k].load<relaxed>().pointer<void>(), a );
m -= a;
a *= element_size;
}
if( m>0 )
copy( my_segment[k].load<relaxed>().pointer<char>() + a, src.my_segment[k].load<relaxed>().pointer<char>() + a, m );
}
__TBB_ASSERT( src.my_early_size==n, "detected use of concurrent_vector::operator= with right side that was concurrently modified" );
}
void* concurrent_vector_base_v3::internal_push_back( size_type element_size, size_type& index ) {
__TBB_ASSERT( sizeof(my_early_size)==sizeof(uintptr_t), NULL );
size_type tmp = my_early_size.fetch_and_increment<acquire>();
index = tmp;
segment_index_t k_old = segment_index_of( tmp );
size_type base = segment_base(k_old);
helper::extend_table_if_necessary(*this, k_old, tmp);
segment_t& s = helper::acquire_segment(*this, k_old, element_size, base==tmp);
size_type j_begin = tmp-base;
return (void*)(s.load<relaxed>().pointer<char>() + element_size*j_begin);
}
void concurrent_vector_base_v3::internal_grow_to_at_least( size_type new_size, size_type element_size, internal_array_op2 init, const void *src ) {
internal_grow_to_at_least_with_result( new_size, element_size, init, src );
}
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_grow_to_at_least_with_result( size_type new_size, size_type element_size, internal_array_op2 init, const void *src ) {
size_type e = my_early_size;
while( e<new_size ) {
size_type f = my_early_size.compare_and_swap(new_size,e);
if( f==e ) {
internal_grow( e, new_size, element_size, init, src );
break;
}
e = f;
}
// Check/wait for segments allocation completes
segment_index_t i, k_old = segment_index_of( new_size-1 );
if( k_old >= pointers_per_short_table && my_segment == my_storage ) {
spin_wait_while_eq( my_segment, my_storage );
}
for( i = 0; i <= k_old; ++i ) {
segment_t &s = my_segment[i];
if(s.load<relaxed>() == segment_not_used()) {
ITT_NOTIFY(sync_prepare, &s);
atomic_backoff backoff(true);
while( my_segment[i].load<acquire>() == segment_not_used() ) // my_segment may change concurrently
backoff.pause();
ITT_NOTIFY(sync_acquired, &s);
}
if( my_segment[i].load<relaxed>() != segment_allocated() )
throw_exception(eid_bad_last_alloc);
}
#if TBB_USE_DEBUG
size_type capacity = internal_capacity();
__TBB_ASSERT( capacity >= new_size, NULL);
#endif
return e;
}
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_grow_by( size_type delta, size_type element_size, internal_array_op2 init, const void *src ) {
size_type result = my_early_size.fetch_and_add(delta);
internal_grow( result, result+delta, element_size, init, src );
return result;
}
void concurrent_vector_base_v3::internal_grow( const size_type start, size_type finish, size_type element_size, internal_array_op2 init, const void *src ) {
__TBB_ASSERT( start<finish, "start must be less than finish" );
segment_index_t k_start = segment_index_of(start), k_end = segment_index_of(finish-1);
helper::assign_first_segment_if_necessary(*this, k_end);
helper::extend_table_if_necessary(*this, k_end, start);
helper range(my_segment, my_first_block, element_size, k_start, start, finish);
for(; k_end > k_start && k_end >= range.first_block; --k_end ) // allocate segments in reverse order
helper::acquire_segment(*this, k_end, element_size, true/*for k_end>k_start*/);
for(; k_start <= k_end; ++k_start ) // but allocate first block in straight order
helper::acquire_segment(*this, k_start, element_size, segment_base( k_start ) >= start );
range.apply( helper::init_body(init, src) );
}
void concurrent_vector_base_v3::internal_resize( size_type n, size_type element_size, size_type max_size, const void *src,
internal_array_op1 destroy, internal_array_op2 init ) {
size_type j = my_early_size;
if( n > j ) { // construct items
internal_reserve(n, element_size, max_size);
my_early_size = n;
helper for_each(my_segment, my_first_block, element_size, segment_index_of(j), j, n);
for_each.apply( helper::safe_init_body(init, src) );
} else {
my_early_size = n;
helper for_each(my_segment, my_first_block, element_size, segment_index_of(n), n, j);
for_each.apply( helper::destroy_body(destroy) );
}
}
concurrent_vector_base_v3::segment_index_t concurrent_vector_base_v3::internal_clear( internal_array_op1 destroy ) {
__TBB_ASSERT( my_segment, NULL );
size_type j = my_early_size;
my_early_size = 0;
helper for_each(my_segment, my_first_block, 0, 0, 0, j); // element_size is safe to be zero if 'start' is zero
j = for_each.apply( helper::destroy_body(destroy) );
size_type i = helper::find_segment_end(*this);
return j < i? i : j+1;
}
void *concurrent_vector_base_v3::internal_compact( size_type element_size, void *table, internal_array_op1 destroy, internal_array_op2 copy )
{
const size_type my_size = my_early_size;
const segment_index_t k_end = helper::find_segment_end(*this); // allocated segments
const segment_index_t k_stop = my_size? segment_index_of(my_size-1) + 1 : 0; // number of segments to store existing items: 0=>0; 1,2=>1; 3,4=>2; [5-8]=>3;..
const segment_index_t first_block = my_first_block; // number of merged segments, getting values from atomics
segment_index_t k = first_block;
if(k_stop < first_block)
k = k_stop;
else
while (k < k_stop && helper::incompact_predicate(segment_size( k ) * element_size) ) k++;
if(k_stop == k_end && k == first_block)
return NULL;
segment_t *const segment_table = my_segment;
internal_segments_table &old = *static_cast<internal_segments_table*>( table );
//this call is left here for sake of backward compatibility, and as a placeholder for table initialization
std::fill_n(old.table,sizeof(old.table)/sizeof(old.table[0]),segment_t());
old.first_block=0;
if ( k != first_block && k ) // first segment optimization
{
// exception can occur here
void *seg = helper::allocate_segment(*this, segment_size(k));
old.table[0].store<relaxed>(seg);
old.first_block = k; // fill info for freeing new segment if exception occurs
// copy items to the new segment
size_type my_segment_size = segment_size( first_block );
for (segment_index_t i = 0, j = 0; i < k && j < my_size; j = my_segment_size) {
__TBB_ASSERT( segment_table[i].load<relaxed>() == segment_allocated(), NULL);
void *s = static_cast<void*>(
static_cast<char*>(seg) + segment_base(i)*element_size );
//TODO: refactor to use std::min
if(j + my_segment_size >= my_size) my_segment_size = my_size - j;
__TBB_TRY { // exception can occur here
copy( s, segment_table[i].load<relaxed>().pointer<void>(), my_segment_size );
} __TBB_CATCH(...) { // destroy all the already copied items
helper for_each(&old.table[0], old.first_block, element_size,
0, 0, segment_base(i)+ my_segment_size);
for_each.apply( helper::destroy_body(destroy) );
__TBB_RETHROW();
}
my_segment_size = i? segment_size( ++i ) : segment_size( i = first_block );
}
// commit the changes
std::copy(segment_table,segment_table + k,old.table);
for (segment_index_t i = 0; i < k; i++) {
segment_table[i].store<relaxed>(static_cast<void*>(
static_cast<char*>(seg) + segment_base(i)*element_size ));
}
old.first_block = first_block; my_first_block = k; // now, first_block != my_first_block
// destroy original copies
my_segment_size = segment_size( first_block ); // old.first_block actually
for (segment_index_t i = 0, j = 0; i < k && j < my_size; j = my_segment_size) {
if(j + my_segment_size >= my_size) my_segment_size = my_size - j;
// destructors are supposed to not throw any exceptions
destroy( old.table[i].load<relaxed>().pointer<void>(), my_segment_size );
my_segment_size = i? segment_size( ++i ) : segment_size( i = first_block );
}
}
// free unnecessary segments allocated by reserve() call
if ( k_stop < k_end ) {
old.first_block = first_block;
std::copy(segment_table+k_stop, segment_table+k_end, old.table+k_stop );
std::fill_n(segment_table+k_stop, (k_end-k_stop), segment_t());
if( !k ) my_first_block = 0;
}
return table;
}
void concurrent_vector_base_v3::internal_swap(concurrent_vector_base_v3& v)
{
size_type my_sz = my_early_size.load<acquire>();
size_type v_sz = v.my_early_size.load<relaxed>();
if(!my_sz && !v_sz) return;
bool my_was_short = (my_segment.load<relaxed>() == my_storage);
bool v_was_short = (v.my_segment.load<relaxed>() == v.my_storage);
//In C++11, this would be: swap(my_storage, v.my_storage);
for (int i=0; i < pointers_per_short_table; ++i){
swap(my_storage[i], v.my_storage[i]);
}
tbb::internal::swap<relaxed>(my_first_block, v.my_first_block);
tbb::internal::swap<relaxed>(my_segment, v.my_segment);
if (my_was_short){
v.my_segment.store<relaxed>(v.my_storage);
}
if(v_was_short){
my_segment.store<relaxed>(my_storage);
}
my_early_size.store<relaxed>(v_sz);
v.my_early_size.store<release>(my_sz);
}
} // namespace internal
} // tbb

View File

@@ -0,0 +1,199 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
#include "tbb/compat/condition_variable"
#include "tbb/atomic.h"
#include "tbb_misc.h"
#include "dynamic_link.h"
#include "itt_notify.h"
namespace tbb {
namespace internal {
//condition_variable
#if _WIN32||_WIN64
using tbb::interface5::internal::condition_variable_using_event;
static atomic<do_once_state> condvar_api_state;
void WINAPI init_condvar_using_event( condition_variable_using_event* cv_event )
{
// TODO: For Metro port, we can always use the API for condition variables, without dynamic_link etc.
cv_event->event = CreateEventEx(NULL, NULL, 0x1 /*CREATE_EVENT_MANUAL_RESET*/, EVENT_ALL_ACCESS );
InitializeCriticalSectionEx( &cv_event->mutex, 4000, 0 );
cv_event->n_waiters = 0;
cv_event->release_count = 0;
cv_event->epoch = 0;
}
BOOL WINAPI sleep_condition_variable_cs_using_event( condition_variable_using_event* cv_event, LPCRITICAL_SECTION cs, DWORD dwMilliseconds )
{
EnterCriticalSection( &cv_event->mutex );
++cv_event->n_waiters;
unsigned my_generation = cv_event->epoch;
LeaveCriticalSection( &cv_event->mutex );
LeaveCriticalSection( cs );
for (;;) {
// should come here at least once
DWORD rc = WaitForSingleObjectEx( cv_event->event, dwMilliseconds, FALSE );
EnterCriticalSection( &cv_event->mutex );
if( rc!=WAIT_OBJECT_0 ) {
--cv_event->n_waiters;
LeaveCriticalSection( &cv_event->mutex );
if( rc==WAIT_TIMEOUT ) {
SetLastError( WAIT_TIMEOUT );
EnterCriticalSection( cs );
}
return false;
}
__TBB_ASSERT( rc==WAIT_OBJECT_0, NULL );
if( cv_event->release_count>0 && cv_event->epoch!=my_generation )
break;
LeaveCriticalSection( &cv_event->mutex );
}
// still in the critical section
--cv_event->n_waiters;
int count = --cv_event->release_count;
LeaveCriticalSection( &cv_event->mutex );
if( count==0 ) {
__TBB_ASSERT( cv_event->event, "Premature destruction of condition variable?" );
ResetEvent( cv_event->event );
}
EnterCriticalSection( cs );
return true;
}
void WINAPI wake_condition_variable_using_event( condition_variable_using_event* cv_event )
{
EnterCriticalSection( &cv_event->mutex );
if( cv_event->n_waiters>cv_event->release_count ) {
SetEvent( cv_event->event ); // Signal the manual-reset event.
++cv_event->release_count;
++cv_event->epoch;
}
LeaveCriticalSection( &cv_event->mutex );
}
void WINAPI wake_all_condition_variable_using_event( condition_variable_using_event* cv_event )
{
EnterCriticalSection( &cv_event->mutex );
if( cv_event->n_waiters>0 ) {
SetEvent( cv_event->event );
cv_event->release_count = cv_event->n_waiters;
++cv_event->epoch;
}
LeaveCriticalSection( &cv_event->mutex );
}
void WINAPI destroy_condvar_using_event( condition_variable_using_event* cv_event )
{
HANDLE my_event = cv_event->event;
EnterCriticalSection( &cv_event->mutex );
// NULL is an invalid HANDLE value
cv_event->event = NULL;
if( cv_event->n_waiters>0 ) {
LeaveCriticalSection( &cv_event->mutex );
spin_wait_until_eq( cv_event->n_waiters, 0 );
// make sure the last thread completes its access to cv
EnterCriticalSection( &cv_event->mutex );
}
LeaveCriticalSection( &cv_event->mutex );
CloseHandle( my_event );
}
void WINAPI destroy_condvar_noop( CONDITION_VARIABLE* /*cv*/ ) { /*no op*/ }
static void (WINAPI *__TBB_init_condvar)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&init_condvar_using_event;
static BOOL (WINAPI *__TBB_condvar_wait)( PCONDITION_VARIABLE, LPCRITICAL_SECTION, DWORD ) = (BOOL (WINAPI *)(PCONDITION_VARIABLE,LPCRITICAL_SECTION, DWORD))&sleep_condition_variable_cs_using_event;
static void (WINAPI *__TBB_condvar_notify_one)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&wake_condition_variable_using_event;
static void (WINAPI *__TBB_condvar_notify_all)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&wake_all_condition_variable_using_event;
static void (WINAPI *__TBB_destroy_condvar)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&destroy_condvar_using_event;
//! Table describing how to link the handlers.
static const dynamic_link_descriptor CondVarLinkTable[] = {
DLD(InitializeConditionVariable, __TBB_init_condvar),
DLD(SleepConditionVariableCS, __TBB_condvar_wait),
DLD(WakeConditionVariable, __TBB_condvar_notify_one),
DLD(WakeAllConditionVariable, __TBB_condvar_notify_all)
};
void init_condvar_module()
{
__TBB_ASSERT( (uintptr_t)__TBB_init_condvar==(uintptr_t)&init_condvar_using_event, NULL );
if( dynamic_link( "Kernel32.dll", CondVarLinkTable, 4 ) )
__TBB_destroy_condvar = (void (WINAPI *)(PCONDITION_VARIABLE))&destroy_condvar_noop;
}
#endif /* _WIN32||_WIN64 */
} // namespace internal
#if _WIN32||_WIN64
namespace interface5 {
namespace internal {
using tbb::internal::condvar_api_state;
using tbb::internal::__TBB_init_condvar;
using tbb::internal::__TBB_condvar_wait;
using tbb::internal::__TBB_condvar_notify_one;
using tbb::internal::__TBB_condvar_notify_all;
using tbb::internal::__TBB_destroy_condvar;
using tbb::internal::init_condvar_module;
void internal_initialize_condition_variable( condvar_impl_t& cv )
{
atomic_do_once( &init_condvar_module, condvar_api_state );
__TBB_init_condvar( &cv.cv_native );
}
void internal_destroy_condition_variable( condvar_impl_t& cv )
{
__TBB_destroy_condvar( &cv.cv_native );
}
void internal_condition_variable_notify_one( condvar_impl_t& cv )
{
__TBB_condvar_notify_one ( &cv.cv_native );
}
void internal_condition_variable_notify_all( condvar_impl_t& cv )
{
__TBB_condvar_notify_all( &cv.cv_native );
}
bool internal_condition_variable_wait( condvar_impl_t& cv, mutex* mtx, const tick_count::interval_t* i )
{
DWORD duration = i ? DWORD((i->seconds()*1000)) : INFINITE;
mtx->set_state( mutex::INITIALIZED );
BOOL res = __TBB_condvar_wait( &cv.cv_native, mtx->native_handle(), duration );
mtx->set_state( mutex::HELD );
return res?true:false;
}
} // namespace internal
} // nameespace interface5
#endif /* _WIN32||_WIN64 */
} // namespace tbb

View File

@@ -0,0 +1,31 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/critical_section.h"
#include "itt_notify.h"
namespace tbb {
namespace internal {
void critical_section_v4::internal_construct() {
ITT_SYNC_CREATE(&my_impl, _T("ppl::critical_section"), _T(""));
}
} // namespace internal
} // namespace tbb

View File

@@ -0,0 +1,133 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_CRITICAL_SECTION_H_
#define _TBB_CRITICAL_SECTION_H_
#if _WIN32||_WIN64
#include "machine/windows_api.h"
#else
#include <pthread.h>
#include <errno.h>
#endif // _WIN32||WIN64
#include "tbb_stddef.h"
#include "tbb_thread.h"
#include "tbb_exception.h"
#include "tbb_profiling.h"
namespace tbb {
namespace internal {
class critical_section_v4 : internal::no_copy {
#if _WIN32||_WIN64
CRITICAL_SECTION my_impl;
#else
pthread_mutex_t my_impl;
#endif
tbb_thread::id my_tid;
public:
void __TBB_EXPORTED_METHOD internal_construct();
critical_section_v4() {
#if _WIN32||_WIN64
InitializeCriticalSectionEx( &my_impl, 4000, 0 );
#else
pthread_mutex_init(&my_impl, NULL);
#endif
internal_construct();
}
~critical_section_v4() {
__TBB_ASSERT(my_tid == tbb_thread::id(), "Destroying a still-held critical section");
#if _WIN32||_WIN64
DeleteCriticalSection(&my_impl);
#else
pthread_mutex_destroy(&my_impl);
#endif
}
class scoped_lock : internal::no_copy {
private:
critical_section_v4 &my_crit;
public:
scoped_lock( critical_section_v4& lock_me) :my_crit(lock_me) {
my_crit.lock();
}
~scoped_lock() {
my_crit.unlock();
}
};
void lock() {
tbb_thread::id local_tid = this_tbb_thread::get_id();
if(local_tid == my_tid) throw_exception( eid_improper_lock );
#if _WIN32||_WIN64
EnterCriticalSection( &my_impl );
#else
int rval = pthread_mutex_lock(&my_impl);
__TBB_ASSERT_EX(!rval, "critical_section::lock: pthread_mutex_lock failed");
#endif
__TBB_ASSERT(my_tid == tbb_thread::id(), NULL);
my_tid = local_tid;
}
bool try_lock() {
bool gotlock;
tbb_thread::id local_tid = this_tbb_thread::get_id();
if(local_tid == my_tid) return false;
#if _WIN32||_WIN64
gotlock = TryEnterCriticalSection( &my_impl ) != 0;
#else
int rval = pthread_mutex_trylock(&my_impl);
// valid returns are 0 (locked) and [EBUSY]
__TBB_ASSERT(rval == 0 || rval == EBUSY, "critical_section::trylock: pthread_mutex_trylock failed");
gotlock = rval == 0;
#endif
if(gotlock) {
my_tid = local_tid;
}
return gotlock;
}
void unlock() {
__TBB_ASSERT(this_tbb_thread::get_id() == my_tid, "thread unlocking critical_section is not thread that locked it");
my_tid = tbb_thread::id();
#if _WIN32||_WIN64
LeaveCriticalSection( &my_impl );
#else
int rval = pthread_mutex_unlock(&my_impl);
__TBB_ASSERT_EX(!rval, "critical_section::unlock: pthread_mutex_unlock failed");
#endif
}
static const bool is_rw_mutex = false;
static const bool is_recursive_mutex = false;
static const bool is_fair_mutex = true;
}; // critical_section_v4
} // namespace internal
typedef internal::critical_section_v4 critical_section;
__TBB_DEFINE_PROFILING_SET_NAME(critical_section)
} // namespace tbb
#endif // _TBB_CRITICAL_SECTION_H_

View File

@@ -0,0 +1,684 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_custom_scheduler_H
#define _TBB_custom_scheduler_H
#include "scheduler.h"
#include "observer_proxy.h"
#include "itt_notify.h"
namespace tbb {
namespace internal {
//! Amount of time to pause between steals.
/** The default values below were found to be best empirically for K-Means
on the 32-way Altix and 4-way (*2 for HT) fxqlin04. */
#ifdef __TBB_STEALING_PAUSE
static const long PauseTime = __TBB_STEALING_PAUSE;
#elif __TBB_ipf
static const long PauseTime = 1500;
#else
static const long PauseTime = 80;
#endif
//------------------------------------------------------------------------
//! Traits classes for scheduler
//------------------------------------------------------------------------
struct DefaultSchedulerTraits {
static const bool itt_possible = true;
static const bool has_slow_atomic = false;
};
struct IntelSchedulerTraits {
static const bool itt_possible = false;
#if __TBB_x86_32||__TBB_x86_64
static const bool has_slow_atomic = true;
#else
static const bool has_slow_atomic = false;
#endif /* __TBB_x86_32||__TBB_x86_64 */
};
//------------------------------------------------------------------------
// custom_scheduler
//------------------------------------------------------------------------
//! A scheduler with a customized evaluation loop.
/** The customization can use SchedulerTraits to make decisions without needing a run-time check. */
template<typename SchedulerTraits>
class custom_scheduler: private generic_scheduler {
typedef custom_scheduler<SchedulerTraits> scheduler_type;
//! Scheduler loop that dispatches tasks.
/** If child is non-NULL, it is dispatched first.
Then, until "parent" has a reference count of 1, other task are dispatched or stolen. */
/*override*/
void local_wait_for_all( task& parent, task* child );
//! Entry point from client code to the scheduler loop that dispatches tasks.
/** The method is virtual, but the *this object is used only for sake of dispatching on the correct vtable,
not necessarily the correct *this object. The correct *this object is looked up in TLS. */
/*override*/
void wait_for_all( task& parent, task* child ) {
static_cast<custom_scheduler*>(governor::local_scheduler())->scheduler_type::local_wait_for_all( parent, child );
}
//! Construct a custom_scheduler
custom_scheduler( arena* a, size_t index ) : generic_scheduler(a, index) {}
//! Decrements ref_count of a predecessor.
/** If it achieves 0, the predecessor is scheduled for execution.
When changing, remember that this is a hot path function. */
void tally_completion_of_predecessor( task& s, task*& bypass_slot ) {
task_prefix& p = s.prefix();
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_releasing, &p.ref_count);
if( SchedulerTraits::has_slow_atomic && p.ref_count==1 )
p.ref_count=0;
else if( __TBB_FetchAndDecrementWrelease(&p.ref_count) > 1 ) {// more references exist
// '__TBB_cl_evict(&p)' degraded performance of parallel_preorder example
return;
}
// Ordering on p.ref_count (superfluous if SchedulerTraits::has_slow_atomic)
__TBB_control_consistency_helper();
__TBB_ASSERT(p.ref_count==0, "completion of task caused predecessor's reference count to underflow");
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_acquired, &p.ref_count);
#if TBB_USE_ASSERT
p.extra_state &= ~es_ref_count_active;
#endif /* TBB_USE_ASSERT */
#if __TBB_RECYCLE_TO_ENQUEUE
if (p.state==task::to_enqueue) {
// related to __TBB_TASK_ARENA TODO: try keep priority of the task
// e.g. rework task_prefix to remember priority of received task and use here
my_arena->enqueue_task(s, 0, my_random );
} else
#endif /*__TBB_RECYCLE_TO_ENQUEUE*/
if( bypass_slot==NULL )
bypass_slot = &s;
else
local_spawn( s, s.prefix().next );
}
public:
static generic_scheduler* allocate_scheduler( arena* a, size_t index ) {
scheduler_type* s = (scheduler_type*)NFS_Allocate(1,sizeof(scheduler_type),NULL);
new( s ) scheduler_type( a, index );
s->assert_task_pool_valid();
ITT_SYNC_CREATE(s, SyncType_Scheduler, SyncObj_TaskPoolSpinning);
return s;
}
//! Try getting a task from the mailbox or stealing from another scheduler.
/** Returns the stolen task or NULL if all attempts fail. */
/* override */ task* receive_or_steal_task( __TBB_atomic reference_count& completion_ref_count, bool return_if_no_work );
}; // class custom_scheduler<>
//------------------------------------------------------------------------
// custom_scheduler methods
//------------------------------------------------------------------------
template<typename SchedulerTraits>
task* custom_scheduler<SchedulerTraits>::receive_or_steal_task( __TBB_atomic reference_count& completion_ref_count,
bool return_if_no_work ) {
task* t = NULL;
bool outermost_dispatch_level = return_if_no_work || master_outermost_level();
bool can_steal_here = can_steal();
my_inbox.set_is_idle( true );
#if __TBB_HOARD_NONLOCAL_TASKS
__TBB_ASSERT(!my_nonlocal_free_list, NULL);
#endif
#if __TBB_TASK_PRIORITY
if ( return_if_no_work && my_arena->my_skipped_fifo_priority ) {
// This thread can dequeue FIFO tasks, and some priority levels of
// FIFO tasks have been bypassed (to prevent deadlock caused by
// dynamic priority changes in nested task group hierarchy).
intptr_t skipped_priority = my_arena->my_skipped_fifo_priority;
if ( my_arena->my_skipped_fifo_priority.compare_and_swap(0, skipped_priority) == skipped_priority &&
skipped_priority > my_arena->my_top_priority )
{
my_market->update_arena_priority( *my_arena, skipped_priority );
}
}
task_stream *ts;
#else /* !__TBB_TASK_PRIORITY */
task_stream *ts = &my_arena->my_task_stream;
#endif /* !__TBB_TASK_PRIORITY */
// TODO: Try to find a place to reset my_limit (under market's lock)
// The number of slots potentially used in the arena. Updated once in a while, as my_limit changes rarely.
size_t n = my_arena->my_limit-1;
int yield_count = 0;
// The state "failure_count==-1" is used only when itt_possible is true,
// and denotes that a sync_prepare has not yet been issued.
for( int failure_count = -static_cast<int>(SchedulerTraits::itt_possible);; ++failure_count) {
__TBB_ASSERT( my_arena->my_limit > 0, NULL );
__TBB_ASSERT( my_arena_index <= n, NULL );
if( completion_ref_count==1 ) {
if( SchedulerTraits::itt_possible ) {
if( failure_count!=-1 ) {
ITT_NOTIFY(sync_prepare, &completion_ref_count);
// Notify Intel(R) Thread Profiler that thread has stopped spinning.
ITT_NOTIFY(sync_acquired, this);
}
ITT_NOTIFY(sync_acquired, &completion_ref_count);
}
__TBB_ASSERT( !t, NULL );
__TBB_control_consistency_helper(); // on ref_count
break; // exit stealing loop and return;
}
// Check if the resource manager requires our arena to relinquish some threads
if ( return_if_no_work && my_arena->my_num_workers_allotted < my_arena->num_workers_active() ) {
#if !__TBB_TASK_ARENA
__TBB_ASSERT( is_worker(), NULL );
#endif
if( SchedulerTraits::itt_possible && failure_count != -1 )
ITT_NOTIFY(sync_cancel, this);
return NULL;
}
#if __TBB_TASK_PRIORITY
ts = &my_arena->my_task_stream[my_arena->my_top_priority];
#endif
// Check if there are tasks mailed to this thread via task-to-thread affinity mechanism.
__TBB_ASSERT(my_affinity_id, NULL);
if ( n && !my_inbox.empty() && (t = get_mailbox_task()) ) {
GATHER_STATISTIC( ++my_counters.mails_received );
}
// Check if there are tasks in starvation-resistant stream.
// Only allowed for workers with empty stack, which is identified by return_if_no_work.
else if ( outermost_dispatch_level && !ts->empty() && (t = ts->pop( my_arena_slot->hint_for_pop)) ) {
ITT_NOTIFY(sync_acquired, ts);
// just proceed with the obtained task
}
#if __TBB_TASK_PRIORITY
// Check if any earlier offloaded non-top priority tasks become returned to the top level
else if ( my_offloaded_tasks && (t=reload_tasks()) ) {
// just proceed with the obtained task
}
#endif /* __TBB_TASK_PRIORITY */
else if ( can_steal_here && n ) {
// Try to steal a task from a random victim.
size_t k = my_random.get() % n;
arena_slot* victim = &my_arena->my_slots[k];
// The following condition excludes the master that might have
// already taken our previous place in the arena from the list .
// of potential victims. But since such a situation can take
// place only in case of significant oversubscription, keeping
// the checks simple seems to be preferable to complicating the code.
if( k >= my_arena_index )
++victim; // Adjusts random distribution to exclude self
task **pool = victim->task_pool;
if( pool == EmptyTaskPool || !(t = steal_task( *victim )) )
goto fail;
if( is_proxy(*t) ) {
task_proxy &tp = *(task_proxy*)t;
t = tp.extract_task<task_proxy::pool_bit>();
if ( !t ) {
// Proxy was empty, so it's our responsibility to free it
free_task<no_cache_small_task>(tp);
goto fail;
}
GATHER_STATISTIC( ++my_counters.proxies_stolen );
}
t->prefix().extra_state |= es_task_is_stolen;
if( is_version_3_task(*t) ) {
my_innermost_running_task = t;
t->prefix().owner = this;
t->note_affinity( my_affinity_id );
}
GATHER_STATISTIC( ++my_counters.steals_committed );
} // end of stealing branch
else
goto fail;
// A task was successfully obtained somewhere
__TBB_ASSERT(t,NULL);
#if __TBB_SCHEDULER_OBSERVER
my_arena->my_observers.notify_entry_observers( my_last_local_observer, is_worker() );
the_global_observer_list.notify_entry_observers( my_last_global_observer, is_worker() );
#endif /* __TBB_SCHEDULER_OBSERVER */
if ( SchedulerTraits::itt_possible && failure_count != -1 ) {
// FIXME - might be victim, or might be selected from a mailbox
// Notify Intel(R) Thread Profiler that thread has stopped spinning.
ITT_NOTIFY(sync_acquired, this);
}
break; // exit stealing loop and return
fail:
GATHER_STATISTIC( ++my_counters.steals_failed );
if( SchedulerTraits::itt_possible && failure_count==-1 ) {
// The first attempt to steal work failed, so notify Intel(R) Thread Profiler that
// the thread has started spinning. Ideally, we would do this notification
// *before* the first failed attempt to steal, but at that point we do not
// know that the steal will fail.
ITT_NOTIFY(sync_prepare, this);
failure_count = 0;
}
// Pause, even if we are going to yield, because the yield might return immediately.
__TBB_Pause(PauseTime);
const int failure_threshold = 2*int(n+1);
if( failure_count>=failure_threshold ) {
#if __TBB_YIELD2P
failure_count = 0;
#else
failure_count = failure_threshold;
#endif
__TBB_Yield();
#if __TBB_TASK_PRIORITY
// Check if there are tasks abandoned by other workers
if ( my_arena->my_orphaned_tasks ) {
// Epoch must be advanced before seizing the list pointer
++my_arena->my_abandonment_epoch;
task* orphans = (task*)__TBB_FetchAndStoreW( &my_arena->my_orphaned_tasks, 0 );
if ( orphans ) {
task** link = NULL;
// Get local counter out of the way (we've just brought in external tasks)
my_local_reload_epoch--;
t = reload_tasks( orphans, link, effective_reference_priority() );
if ( orphans ) {
*link = my_offloaded_tasks;
if ( !my_offloaded_tasks )
my_offloaded_task_list_tail_link = link;
my_offloaded_tasks = orphans;
}
__TBB_ASSERT( !my_offloaded_tasks == !my_offloaded_task_list_tail_link, NULL );
if ( t ) {
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_cancel, this);
break; // exit stealing loop and return
}
}
}
#endif /* __TBB_TASK_PRIORITY */
const int yield_threshold = 100;
if( yield_count++ >= yield_threshold ) {
// When a worker thread has nothing to do, return it to RML.
// For purposes of affinity support, the thread is considered idle while in RML.
#if __TBB_TASK_PRIORITY
if( return_if_no_work || my_arena->my_top_priority > my_arena->my_bottom_priority ) {
if ( my_arena->is_out_of_work() && return_if_no_work ) {
#else /* !__TBB_TASK_PRIORITY */
if ( return_if_no_work && my_arena->is_out_of_work() ) {
#endif /* !__TBB_TASK_PRIORITY */
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_cancel, this);
return NULL;
}
#if __TBB_TASK_PRIORITY
}
if ( my_offloaded_tasks ) {
// Safeguard against any sloppiness in managing reload epoch
// counter (e.g. on the hot path because of performance reasons).
my_local_reload_epoch--;
// Break the deadlock caused by a higher priority dispatch loop
// stealing and offloading a lower priority task. Priority check
// at the stealing moment cannot completely preclude such cases
// because priorities can changes dynamically.
if ( !return_if_no_work && *my_ref_top_priority > my_arena->my_top_priority ) {
GATHER_STATISTIC( ++my_counters.prio_ref_fixups );
my_ref_top_priority = &my_arena->my_top_priority;
// it's expected that only outermost workers can use global reload epoch
__TBB_ASSERT(!worker_outermost_level(), NULL);
__TBB_ASSERT(my_ref_reload_epoch == &my_arena->my_reload_epoch, NULL);
}
}
#endif /* __TBB_TASK_PRIORITY */
} // end of arena snapshot branch
// If several attempts did not find work, re-read the arena limit.
n = my_arena->my_limit-1;
} // end of yielding branch
} // end of nonlocal task retrieval loop
my_inbox.set_is_idle( false );
return t;
}
template<typename SchedulerTraits>
void custom_scheduler<SchedulerTraits>::local_wait_for_all( task& parent, task* child ) {
__TBB_ASSERT( governor::is_set(this), NULL );
__TBB_ASSERT( parent.ref_count() >= (child && child->parent() == &parent ? 2 : 1), "ref_count is too small" );
assert_task_pool_valid();
// Using parent's refcount in sync_prepare (in the stealing loop below) is
// a workaround for TP. We need to name it here to display correctly in Ampl.
if( SchedulerTraits::itt_possible )
ITT_SYNC_CREATE(&parent.prefix().ref_count, SyncType_Scheduler, SyncObj_TaskStealingLoop);
#if __TBB_TASK_GROUP_CONTEXT
__TBB_ASSERT( parent.prefix().context || (is_worker() && &parent == my_dummy_task), "parent task does not have context" );
#endif /* __TBB_TASK_GROUP_CONTEXT */
task* t = child;
// Constant all_local_work_done is an unreachable refcount value that prevents
// early quitting the dispatch loop. It is defined to be in the middle of the range
// of negative values representable by the reference_count type.
static const reference_count
// For normal dispatch loops
parents_work_done = 1,
// For termination dispatch loops in masters
all_local_work_done = (reference_count)3 << (sizeof(reference_count) * 8 - 2);
reference_count quit_point;
#if __TBB_TASK_PRIORITY
__TBB_ASSERT( (uintptr_t)*my_ref_top_priority < (uintptr_t)num_priority_levels, NULL );
volatile intptr_t *old_ref_top_priority = my_ref_top_priority;
// When entering nested parallelism level market level counter
// must be replaced with the one local to this arena.
volatile uintptr_t *old_ref_reload_epoch = my_ref_reload_epoch;
#endif /* __TBB_TASK_PRIORITY */
task* old_dispatching_task = my_dispatching_task;
my_dispatching_task = my_innermost_running_task;
if( master_outermost_level() ) {
// We are in the outermost task dispatch loop of a master thread or a worker which mimics master
__TBB_ASSERT( !is_worker() || my_dispatching_task != old_dispatching_task, NULL );
quit_point = &parent == my_dummy_task ? all_local_work_done : parents_work_done;
} else {
quit_point = parents_work_done;
#if __TBB_TASK_PRIORITY
if ( &parent != my_dummy_task ) {
// We are in a nested dispatch loop.
// Market or arena priority must not prevent child tasks from being
// executed so that dynamic priority changes did not cause deadlock.
my_ref_top_priority = &parent.prefix().context->my_priority;
my_ref_reload_epoch = &my_arena->my_reload_epoch;
if(my_ref_reload_epoch != old_ref_reload_epoch)
my_local_reload_epoch = *my_ref_reload_epoch-1;
}
#endif /* __TBB_TASK_PRIORITY */
}
cpu_ctl_env_helper cpu_ctl_helper;
if ( t )
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
#if TBB_USE_EXCEPTIONS
// Infinite safeguard EH loop
for (;;) {
try {
#endif /* TBB_USE_EXCEPTIONS */
// Outer loop receives tasks from global environment (via mailbox, FIFO queue(s),
// and by stealing from other threads' task pools).
// All exit points from the dispatch loop are located in its immediate scope.
for(;;) {
// Middle loop retrieves tasks from the local task pool.
for(;;) {
// Inner loop evaluates tasks coming from nesting loops and those returned
// by just executed tasks (bypassing spawn or enqueue calls).
while(t) {
__TBB_ASSERT( my_inbox.is_idle_state(false), NULL );
__TBB_ASSERT(!is_proxy(*t),"unexpected proxy");
__TBB_ASSERT( t->prefix().owner, NULL );
assert_task_valid(*t);
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_ASSERT
assert_context_valid(t->prefix().context);
if ( !t->prefix().context->my_cancellation_requested )
#endif
__TBB_ASSERT( 1L<<t->state() & (1L<<task::allocated|1L<<task::ready|1L<<task::reexecute), NULL );
assert_task_pool_valid();
#if __TBB_TASK_PRIORITY
intptr_t p = priority(*t);
if ( p != *my_ref_top_priority && (t->prefix().extra_state & es_task_enqueued) == 0) {
assert_priority_valid(p);
if ( p != my_arena->my_top_priority ) {
my_market->update_arena_priority( *my_arena, p );
}
if ( p < effective_reference_priority() ) {
if ( !my_offloaded_tasks ) {
my_offloaded_task_list_tail_link = &t->prefix().next_offloaded;
// Erase possible reference to the owner scheduler (next_offloaded is a union member)
*my_offloaded_task_list_tail_link = NULL;
}
offload_task( *t, p );
if ( in_arena() ) {
t = winnow_task_pool();
if ( t )
continue;
}
else {
// Mark arena as full to unlock arena priority level adjustment
// by arena::is_out_of_work(), and ensure worker's presence.
my_arena->advertise_new_work<false>();
}
goto stealing_ground;
}
}
#endif /* __TBB_TASK_PRIORITY */
task* t_next = NULL;
my_innermost_running_task = t;
t->prefix().owner = this;
t->prefix().state = task::executing;
#if __TBB_TASK_GROUP_CONTEXT
if ( !t->prefix().context->my_cancellation_requested )
#endif
{
GATHER_STATISTIC( ++my_counters.tasks_executed );
GATHER_STATISTIC( my_counters.avg_arena_concurrency += my_arena->num_workers_active() );
GATHER_STATISTIC( my_counters.avg_assigned_workers += my_arena->my_num_workers_allotted );
#if __TBB_TASK_PRIORITY
GATHER_STATISTIC( my_counters.avg_arena_prio += p );
GATHER_STATISTIC( my_counters.avg_market_prio += my_market->my_global_top_priority );
#endif /* __TBB_TASK_PRIORITY */
ITT_STACK(SchedulerTraits::itt_possible, callee_enter, t->prefix().context->itt_caller);
t_next = t->execute();
ITT_STACK(SchedulerTraits::itt_possible, callee_leave, t->prefix().context->itt_caller);
if (t_next) {
__TBB_ASSERT( t_next->state()==task::allocated,
"if task::execute() returns task, it must be marked as allocated" );
reset_extra_state(t_next);
#if TBB_USE_ASSERT
affinity_id next_affinity=t_next->prefix().affinity;
if (next_affinity != 0 && next_affinity != my_affinity_id)
GATHER_STATISTIC( ++my_counters.affinity_ignored );
#endif
}
}
assert_task_pool_valid();
switch( t->state() ) {
case task::executing: {
task* s = t->parent();
__TBB_ASSERT( my_innermost_running_task==t, NULL );
__TBB_ASSERT( t->prefix().ref_count==0, "Task still has children after it has been executed" );
t->~task();
if( s )
tally_completion_of_predecessor(*s, t_next);
free_task<no_hint>( *t );
assert_task_pool_valid();
break;
}
case task::recycle: // set by recycle_as_safe_continuation()
t->prefix().state = task::allocated;
#if __TBB_RECYCLE_TO_ENQUEUE
case task::to_enqueue: // set by recycle_to_enqueue()
#endif
__TBB_ASSERT( t_next != t, "a task returned from method execute() can not be recycled in another way" );
reset_extra_state(t);
// for safe continuation, need atomically decrement ref_count;
tally_completion_of_predecessor(*t, t_next);
assert_task_pool_valid();
break;
case task::reexecute: // set by recycle_to_reexecute()
__TBB_ASSERT( t_next, "reexecution requires that method execute() return another task" );
__TBB_ASSERT( t_next != t, "a task returned from method execute() can not be recycled in another way" );
t->prefix().state = task::allocated;
reset_extra_state(t);
local_spawn( *t, t->prefix().next );
assert_task_pool_valid();
break;
case task::allocated:
reset_extra_state(t);
break;
#if TBB_USE_ASSERT
case task::ready:
__TBB_ASSERT( false, "task is in READY state upon return from method execute()" );
break;
default:
__TBB_ASSERT( false, "illegal state" );
#else
default: // just to shut up some compilation warnings
break;
#endif /* TBB_USE_ASSERT */
}
GATHER_STATISTIC( t_next ? ++my_counters.spawns_bypassed : 0 );
t = t_next;
} // end of scheduler bypass loop
assert_task_pool_valid();
if ( parent.prefix().ref_count == quit_point ) {
__TBB_ASSERT( quit_point != all_local_work_done, NULL );
__TBB_control_consistency_helper(); // on ref_count
ITT_NOTIFY(sync_acquired, &parent.prefix().ref_count);
goto done;
}
if ( in_arena() ) {
t = get_task();
}
else {
__TBB_ASSERT( is_quiescent_local_task_pool_reset(), NULL );
break;
}
__TBB_ASSERT(!t || !is_proxy(*t),"unexpected proxy");
assert_task_pool_valid();
if ( !t ) break;
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
}; // end of local task pool retrieval loop
#if __TBB_TASK_PRIORITY
stealing_ground:
#endif /* __TBB_TASK_PRIORITY */
#if __TBB_HOARD_NONLOCAL_TASKS
// before stealing, previously stolen task objects are returned
for (; my_nonlocal_free_list; my_nonlocal_free_list = t ) {
t = my_nonlocal_free_list->prefix().next;
free_nonlocal_small_task( *my_nonlocal_free_list );
}
#endif
if ( quit_point == all_local_work_done ) {
__TBB_ASSERT( !in_arena() && is_quiescent_local_task_pool_reset(), NULL );
__TBB_ASSERT( !worker_outermost_level(), NULL );
my_innermost_running_task = my_dispatching_task;
my_dispatching_task = old_dispatching_task;
#if __TBB_TASK_PRIORITY
my_ref_top_priority = old_ref_top_priority;
if(my_ref_reload_epoch != old_ref_reload_epoch)
my_local_reload_epoch = *old_ref_reload_epoch-1;
my_ref_reload_epoch = old_ref_reload_epoch;
#endif /* __TBB_TASK_PRIORITY */
return;
}
// The following assertion may be falsely triggered in the presence of enqueued tasks
//__TBB_ASSERT( my_arena->my_max_num_workers > 0 || my_market->my_ref_count > 1
// || parent.prefix().ref_count == 1, "deadlock detected" );
// Dispatching task pointer is NULL *iff* this is a worker thread in its outermost
// dispatch loop (i.e. its execution stack is empty). In this case it should exit it
// either when there is no more work in the current arena, or when revoked by the market.
t = receive_or_steal_task( parent.prefix().ref_count, worker_outermost_level() );
if ( !t )
goto done;
__TBB_ASSERT(!is_proxy(*t),"unexpected proxy");
// The user can capture another the FPU settings to the context so the
// cached data in the helper can be out-of-date and we cannot do fast
// check.
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
} // end of infinite stealing loop
#if TBB_USE_EXCEPTIONS
__TBB_ASSERT( false, "Must never get here" );
} // end of try-block
TbbCatchAll( t->prefix().context );
// Complete post-processing ...
if( t->state() == task::recycle
#if __TBB_RECYCLE_TO_ENQUEUE
// TODO: the enqueue semantics gets lost below, consider reimplementing
|| t->state() == task::to_enqueue
#endif
) {
// ... for recycled tasks to atomically decrement ref_count
t->prefix().state = task::allocated;
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_releasing, &t->prefix().ref_count);
if( __TBB_FetchAndDecrementWrelease(&t->prefix().ref_count)==1 ) {
if( SchedulerTraits::itt_possible )
ITT_NOTIFY(sync_acquired, &t->prefix().ref_count);
}else{
t = NULL;
}
}
} // end of infinite EH loop
__TBB_ASSERT( false, "Must never get here too" );
#endif /* TBB_USE_EXCEPTIONS */
done:
my_innermost_running_task = my_dispatching_task;
my_dispatching_task = old_dispatching_task;
#if __TBB_TASK_PRIORITY
my_ref_top_priority = old_ref_top_priority;
if(my_ref_reload_epoch != old_ref_reload_epoch)
my_local_reload_epoch = *old_ref_reload_epoch-1;
my_ref_reload_epoch = old_ref_reload_epoch;
#endif /* __TBB_TASK_PRIORITY */
if ( !ConcurrentWaitsEnabled(parent) ) {
if ( parent.prefix().ref_count != parents_work_done ) {
// This is a worker that was revoked by the market.
#if __TBB_TASK_ARENA
__TBB_ASSERT( worker_outermost_level(),
"Worker thread exits nested dispatch loop prematurely" );
#else
__TBB_ASSERT( is_worker() && worker_outermost_level(),
"Worker thread exits nested dispatch loop prematurely" );
#endif
return;
}
parent.prefix().ref_count = 0;
}
#if TBB_USE_ASSERT
parent.prefix().extra_state &= ~es_ref_count_active;
#endif /* TBB_USE_ASSERT */
#if __TBB_TASK_GROUP_CONTEXT
__TBB_ASSERT(parent.prefix().context && default_context(), NULL);
task_group_context* parent_ctx = parent.prefix().context;
if ( parent_ctx->my_cancellation_requested ) {
task_group_context::exception_container_type *pe = parent_ctx->my_exception;
if ( master_outermost_level() && parent_ctx == default_context() ) {
// We are in the outermost task dispatch loop of a master thread, and
// the whole task tree has been collapsed. So we may clear cancellation data.
parent_ctx->my_cancellation_requested = 0;
// TODO: Add assertion that master's dummy task context does not have children
parent_ctx->my_state &= ~(uintptr_t)task_group_context::may_have_children;
}
if ( pe ) {
// On Windows, FPU control settings changed in the helper destructor are not visible
// outside a catch block. So restore the default settings manually before rethrowing
// the exception.
cpu_ctl_helper.restore_default();
pe->throw_self();
}
}
__TBB_ASSERT(!is_worker() || !CancellationInfoPresent(*my_dummy_task),
"Worker's dummy task context modified");
__TBB_ASSERT(!master_outermost_level() || !CancellationInfoPresent(*my_dummy_task),
"Unexpected exception or cancellation data in the master's dummy task");
#endif /* __TBB_TASK_GROUP_CONTEXT */
assert_task_pool_valid();
}
} // namespace internal
} // namespace tbb
#endif /* _TBB_custom_scheduler_H */

View File

@@ -0,0 +1,560 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "dynamic_link.h"
#include "tbb/tbb_config.h"
/*
This file is used by both TBB and OpenMP RTL. Do not use __TBB_ASSERT() macro
and runtime_warning() function because they are not available in OpenMP. Use
LIBRARY_ASSERT and DYNAMIC_LINK_WARNING instead.
*/
#include <cstdarg> // va_list etc.
#if _WIN32
#include <malloc.h>
// Unify system calls
#define dlopen( name, flags ) LoadLibraryA( name )
#define dlsym( handle, name ) GetProcAddress( handle, name )
#define dlclose( handle ) ( ! FreeLibrary( handle ) )
#define dlerror() GetLastError()
#ifndef PATH_MAX
#define PATH_MAX MAX_PATH
#endif
#else /* _WIN32 */
#include <dlfcn.h>
#include <string.h>
#include <unistd.h>
#include <limits.h>
#include <stdlib.h>
#endif /* _WIN32 */
#if __TBB_WEAK_SYMBOLS_PRESENT
//TODO: use function attribute for weak symbols instead of the pragma.
#pragma weak dlopen
#pragma weak dlsym
#pragma weak dlclose
#pragma weak dlerror
#pragma weak dladdr
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
#include "tbb/tbb_misc.h"
#define __USE_TBB_ATOMICS ( !(__linux__&&__ia64__) || __TBB_BUILD )
#define __USE_STATIC_DL_INIT (!__ANDROID__)
#if !__USE_TBB_ATOMICS
#include <pthread.h>
#endif
/*
dynamic_link is a common interface for searching for required symbols in an
executable and dynamic libraries.
dynamic_link provides certain guarantees:
1. Either all or none of the requested symbols are resolved. Moreover, if
symbols are not resolved, the dynamic_link_descriptor table is not modified;
2. All returned symbols have secured life time: this means that none of them
can be invalidated until dynamic_unlink is called;
3. Any loaded library is loaded only via the full path. The full path is that
from which the runtime itself was loaded. (This is done to avoid security
issues caused by loading libraries from insecure paths).
dynamic_link searches for the requested symbols in three stages, stopping as
soon as all of the symbols have been resolved.
1. Search the global scope:
a. On Windows: dynamic_link tries to obtain the handle of the requested
library and if it succeeds it resolves the symbols via that handle.
b. On Linux: dynamic_link tries to search for the symbols in the global
scope via the main program handle. If the symbols are present in the global
scope their life time is not guaranteed (since dynamic_link does not know
anything about the library from which they are exported). Therefore it
tries to "pin" the symbols by obtaining the library name and reopening it.
dlopen may fail to reopen the library in two cases:
i. The symbols are exported from the executable. Currently dynamic _link
cannot handle this situation, so it will not find these symbols in this
step.
ii. The necessary library has been unloaded and cannot be reloaded. It
seems there is nothing that can be done in this case. No symbols are
returned.
2. Dynamic load: an attempt is made to load the requested library via the
full path.
The full path used is that from which the runtime itself was loaded. If the
library can be loaded, then an attempt is made to resolve the requested
symbols in the newly loaded library.
If the symbols are not found the library is unloaded.
3. Weak symbols: if weak symbols are available they are returned.
*/
OPEN_INTERNAL_NAMESPACE
#if __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED
#if !defined(DYNAMIC_LINK_WARNING) && !__TBB_WIN8UI_SUPPORT
// Report runtime errors and continue.
#define DYNAMIC_LINK_WARNING dynamic_link_warning
static void dynamic_link_warning( dynamic_link_error_t code, ... ) {
(void) code;
} // library_warning
#endif /* DYNAMIC_LINK_WARNING */
static bool resolve_symbols( dynamic_link_handle module, const dynamic_link_descriptor descriptors[], size_t required )
{
LIBRARY_ASSERT( module != NULL, "Module handle is NULL" );
if ( module == NULL )
return false;
#if __TBB_WEAK_SYMBOLS_PRESENT
if ( !dlsym ) return false;
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
const size_t n_desc=20; // Usually we don't have more than 20 descriptors per library
LIBRARY_ASSERT( required <= n_desc, "Too many descriptors is required" );
if ( required > n_desc ) return false;
pointer_to_handler h[n_desc];
for ( size_t k = 0; k < required; ++k ) {
dynamic_link_descriptor const & desc = descriptors[k];
pointer_to_handler addr = (pointer_to_handler)dlsym( module, desc.name );
if ( !addr ) {
return false;
}
h[k] = addr;
}
// Commit the entry points.
// Cannot use memset here, because the writes must be atomic.
for( size_t k = 0; k < required; ++k )
*descriptors[k].handler = h[k];
return true;
}
#if __TBB_WIN8UI_SUPPORT
bool dynamic_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required, dynamic_link_handle*, int flags ) {
dynamic_link_handle tmp_handle = NULL;
TCHAR wlibrary[256];
if ( MultiByteToWideChar(CP_UTF8, 0, library, -1, wlibrary, 255) == 0 ) return false;
if ( flags & DYNAMIC_LINK_LOAD )
tmp_handle = LoadPackagedLibrary( wlibrary, 0 );
if (tmp_handle != NULL){
return resolve_symbols(tmp_handle, descriptors, required);
}else{
return false;
}
}
void dynamic_unlink( dynamic_link_handle ) {
}
void dynamic_unlink_all() {
}
#else
/*
There is a security issue on Windows: LoadLibrary() may load and execute malicious code.
See http://www.microsoft.com/technet/security/advisory/2269637.mspx for details.
To avoid the issue, we have to pass full path (not just library name) to LoadLibrary. This
function constructs full path to the specified library (it is assumed the library located
side-by-side with the tbb.dll.
The function constructs absolute path for given relative path. Important: Base directory is not
current one, it is the directory tbb.dll loaded from.
Example:
Let us assume "tbb.dll" is located in "c:\program files\common\intel\" directory, e. g.
absolute path of tbb library is "c:\program files\common\intel\tbb.dll". Absolute path for
"tbbmalloc.dll" would be "c:\program files\common\intel\tbbmalloc.dll". Absolute path for
"malloc\tbbmalloc.dll" would be "c:\program files\common\intel\malloc\tbbmalloc.dll".
*/
// Struct handle_storage is used by dynamic_link routine to store handles of
// all loaded or pinned dynamic libraries. When TBB is shut down, it calls
// dynamic_unlink_all() that unloads modules referenced by handle_storage.
// This struct should not have any constructors since it may be used before
// the constructor is called.
#define MAX_LOADED_MODULES 8 // The number of maximum possible modules which can be loaded
struct handle_storage {
#if __USE_TBB_ATOMICS
::tbb::atomic<size_t> my_size;
#else
size_t my_size;
pthread_spinlock_t my_lock;
#endif
dynamic_link_handle my_handles[MAX_LOADED_MODULES];
void add_handle(const dynamic_link_handle &handle) {
#if !__USE_TBB_ATOMICS
int res = pthread_spin_lock( &my_lock );
LIBRARY_ASSERT( res==0, "pthread_spin_lock failed" );
#endif
const size_t ind = my_size++;
#if !__USE_TBB_ATOMICS
res = pthread_spin_unlock( &my_lock );
LIBRARY_ASSERT( res==0, "pthread_spin_unlock failed" );
#endif
LIBRARY_ASSERT( ind < MAX_LOADED_MODULES, "Too many modules are loaded" );
my_handles[ind] = handle;
}
void free_handles() {
const size_t size = my_size;
for (size_t i=0; i<size; ++i)
dynamic_unlink( my_handles[i] );
}
};
handle_storage handles;
#if __USE_TBB_ATOMICS
static void atomic_once ( void (*func) (void), tbb::atomic< tbb::internal::do_once_state > &once_state ) {
tbb::internal::atomic_do_once( func, once_state );
}
#define ATOMIC_ONCE_DECL( var ) tbb::atomic< tbb::internal::do_once_state > var
#else
static void atomic_once ( void (*func) (), pthread_once_t &once_state ) {
pthread_once( &once_state, func );
}
#define ATOMIC_ONCE_DECL( var ) pthread_once_t var = PTHREAD_ONCE_INIT
#endif
ATOMIC_ONCE_DECL( init_dl_data_state );
static struct _ap_data {
char _path[PATH_MAX+1];
size_t _len;
} ap_data;
static void init_ap_data() {
#if _WIN32
// Get handle of our DLL first.
HMODULE handle;
BOOL brc = GetModuleHandleExA(
GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
(LPSTR)( & dynamic_link ), // any function inside the library can be used for the address
& handle
);
if ( !brc ) { // Error occurred.
int err = GetLastError();
DYNAMIC_LINK_WARNING( dl_sys_fail, "GetModuleHandleEx", err );
return;
}
// Now get path to our DLL.
DWORD drc = GetModuleFileNameA( handle, ap_data._path, static_cast< DWORD >( PATH_MAX ) );
if ( drc == 0 ) { // Error occurred.
int err = GetLastError();
DYNAMIC_LINK_WARNING( dl_sys_fail, "GetModuleFileName", err );
return;
}
if ( drc >= PATH_MAX ) { // Buffer too short.
DYNAMIC_LINK_WARNING( dl_buff_too_small );
return;
}
// Find the position of the last backslash.
char *backslash = strrchr( ap_data._path, '\\' );
if ( !backslash ) { // Backslash not found.
LIBRARY_ASSERT( backslash!=NULL, "Unbelievable.");
return;
}
LIBRARY_ASSERT( backslash >= ap_data._path, "Unbelievable.");
ap_data._len = (size_t)(backslash - ap_data._path) + 1;
*(backslash+1) = 0;
#else
// Get the library path
#if __TBB_WEAK_SYMBOLS_PRESENT
if ( !dladdr || !dlerror ) return;
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
Dl_info dlinfo;
int res = dladdr( (void*)&dynamic_link, &dlinfo ); // any function inside the library can be used for the address
if ( !res ) {
char const * err = dlerror();
DYNAMIC_LINK_WARNING( dl_sys_fail, "dladdr", err );
return;
} else {
LIBRARY_ASSERT( dlinfo.dli_fname!=NULL, "Unbelievable." );
}
char const *slash = strrchr( dlinfo.dli_fname, '/' );
size_t fname_len=0;
if ( slash ) {
LIBRARY_ASSERT( slash >= dlinfo.dli_fname, "Unbelievable.");
fname_len = (size_t)(slash - dlinfo.dli_fname) + 1;
}
size_t rc;
if ( dlinfo.dli_fname[0]=='/' ) {
// The library path is absolute
rc = 0;
ap_data._len = 0;
} else {
// The library path is relative so get the current working directory
if ( !getcwd( ap_data._path, sizeof(ap_data._path)/sizeof(ap_data._path[0]) ) ) {
DYNAMIC_LINK_WARNING( dl_buff_too_small );
return;
}
ap_data._len = strlen( ap_data._path );
ap_data._path[ap_data._len++]='/';
rc = ap_data._len;
}
if ( fname_len>0 ) {
if ( ap_data._len>PATH_MAX ) {
DYNAMIC_LINK_WARNING( dl_buff_too_small );
ap_data._len=0;
return;
}
strncpy( ap_data._path+rc, dlinfo.dli_fname, fname_len );
ap_data._len += fname_len;
ap_data._path[ap_data._len]=0;
}
#endif /* _WIN32 */
}
static void init_dl_data() {
init_ap_data();
#if !__USE_TBB_ATOMICS
int res;
res = pthread_spin_init( &handles.my_lock, PTHREAD_PROCESS_SHARED );
LIBRARY_ASSERT( res==0, "pthread_spin_init failed" );
#endif
}
// ap_data structure is initialized with current directory on Linux.
// So it should be initialized as soon as possible since the current directory may be changed.
// static_init_ap_data object provides this initialization during library loading.
static class _static_init_dl_data {
public:
_static_init_dl_data() {
#if __USE_STATIC_DL_INIT
atomic_once( &init_dl_data, init_dl_data_state );
#endif
}
#if !__USE_TBB_ATOMICS
~_static_init_dl_data() {
int res;
res = pthread_spin_destroy( &handles.my_lock );
LIBRARY_ASSERT( res==0, "pthread_spin_destroy failed" );
}
#endif
} static_init_dl_data;
/*
The function constructs absolute path for given relative path. Important: Base directory is not
current one, it is the directory libtbb.so loaded from.
Arguments:
in name -- Name of a file (may be with relative path; it must not be an absolute one).
out path -- Buffer to save result (absolute path) to.
in len -- Size of buffer.
ret -- 0 -- Error occurred.
> len -- Buffer too short, required size returned.
otherwise -- Ok, number of characters (not counting terminating null) written to
buffer.
*/
#if __TBB_DYNAMIC_LOAD_ENABLED
static size_t abs_path( char const * name, char * path, size_t len ) {
atomic_once( &init_dl_data, init_dl_data_state );
if ( !ap_data._len )
return 0;
size_t name_len = strlen( name );
size_t full_len = name_len+ap_data._len;
if ( full_len < len ) {
strncpy( path, ap_data._path, ap_data._len );
strncpy( path+ap_data._len, name, name_len );
path[full_len] = 0;
}
return full_len;
}
#endif // __TBB_DYNAMIC_LOAD_ENABLED
#if __TBB_WEAK_SYMBOLS_PRESENT
static bool weak_symbol_link( const dynamic_link_descriptor descriptors[], size_t required )
{
// Check if the required entries are present in what was loaded into our process.
for ( size_t k = 0; k < required; ++k )
if ( !descriptors[k].ptr )
return false;
// Commit the entry points.
for ( size_t k = 0; k < required; ++k )
*descriptors[k].handler = (pointer_to_handler) descriptors[k].ptr;
return true;
}
#else
static bool weak_symbol_link( const dynamic_link_descriptor[], size_t ) {
return false;
}
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
void dynamic_unlink( dynamic_link_handle handle ) {
if ( handle ) {
#if __TBB_WEAK_SYMBOLS_PRESENT
LIBRARY_ASSERT( dlclose != NULL, "dlopen is present but dlclose is NOT present!?" );
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
#if __TBB_DYNAMIC_LOAD_ENABLED
dlclose( handle );
#endif /* __TBB_DYNAMIC_LOAD_ENABLED */
}
}
void dynamic_unlink_all() {
handles.free_handles();
}
#if _WIN32
static dynamic_link_handle global_symbols_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required ) {
dynamic_link_handle library_handle;
if ( GetModuleHandleExA( 0, library, &library_handle ) ) {
if ( resolve_symbols( library_handle, descriptors, required ) )
return library_handle;
else
FreeLibrary( library_handle );
}
return 0;
}
#else /* _WIN32 */
// It is supposed that all symbols are from the only one library
static dynamic_link_handle pin_symbols( dynamic_link_descriptor desc, const dynamic_link_descriptor descriptors[], size_t required ) {
// The library has been loaded by another module and contains at least one requested symbol.
// But after we obtained the symbol the library can be unloaded by another thread
// invalidating our symbol. Therefore we need to pin the library in memory.
dynamic_link_handle library_handle;
Dl_info info;
// Get library's name from earlier found symbol
if ( dladdr( (void*)*desc.handler, &info ) ) {
// Pin the library
library_handle = dlopen( info.dli_fname, RTLD_LAZY );
if ( library_handle ) {
// If original library was unloaded before we pinned it
// and then another module loaded in its place, the earlier
// found symbol would become invalid. So revalidate them.
if ( !resolve_symbols( library_handle, descriptors, required ) ) {
// Wrong library.
dynamic_unlink(library_handle);
library_handle = 0;
}
} else {
char const * err = dlerror();
DYNAMIC_LINK_WARNING( dl_lib_not_found, info.dli_fname, err );
}
}
else {
// The library have been unloaded by another thread
library_handle = 0;
}
return library_handle;
}
static dynamic_link_handle global_symbols_link( const char*, const dynamic_link_descriptor descriptors[], size_t required ) {
#if __TBB_WEAK_SYMBOLS_PRESENT
if ( !dlopen ) return 0;
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
dynamic_link_handle library_handle = dlopen( NULL, RTLD_LAZY );
#if __ANDROID__
// On Android dlopen( NULL ) returns NULL if it is called during dynamic module initialization.
if ( !library_handle )
return 0;
#endif
// Check existence of only the first symbol, then use it to find the library and load all necessary symbols
pointer_to_handler handler;
dynamic_link_descriptor desc = { descriptors[0].name, &handler };
if ( resolve_symbols( library_handle, &desc, 1 ) )
return pin_symbols( desc, descriptors, required );
return 0;
}
#endif /* _WIN32 */
static void save_library_handle( dynamic_link_handle src, dynamic_link_handle *dst ) {
if ( dst )
*dst = src;
else
handles.add_handle( src );
}
dynamic_link_handle dynamic_load( const char* library, const dynamic_link_descriptor descriptors[], size_t required ) {
#if __TBB_DYNAMIC_LOAD_ENABLED
#if _XBOX
return LoadLibrary (library);
#else /* _XBOX */
size_t const len = PATH_MAX + 1;
char path[ len ];
size_t rc = abs_path( library, path, len );
if ( 0 < rc && rc < len ) {
#if _WIN32
// Prevent Windows from displaying silly message boxes if it fails to load library
// (e.g. because of MS runtime problems - one of those crazy manifest related ones)
UINT prev_mode = SetErrorMode (SEM_FAILCRITICALERRORS);
#endif /* _WIN32 */
#if __TBB_WEAK_SYMBOLS_PRESENT
if ( !dlopen ) return 0;
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
dynamic_link_handle library_handle = dlopen( path, RTLD_LAZY );
#if _WIN32
SetErrorMode (prev_mode);
#endif /* _WIN32 */
if( library_handle ) {
if( !resolve_symbols( library_handle, descriptors, required ) ) {
// The loaded library does not contain all the expected entry points
dynamic_unlink( library_handle );
library_handle = NULL;
}
} else
DYNAMIC_LINK_WARNING( dl_lib_not_found, path, dlerror() );
return library_handle;
} else if ( rc>=len )
DYNAMIC_LINK_WARNING( dl_buff_too_small );
// rc == 0 means failing of init_ap_data so the warning has already been issued.
#endif /* _XBOX */
#endif /* __TBB_DYNAMIC_LOAD_ENABLED */
return 0;
}
bool dynamic_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required, dynamic_link_handle *handle, int flags ) {
// TODO: May global_symbols_link find weak symbols?
dynamic_link_handle library_handle = ( flags & DYNAMIC_LINK_GLOBAL ) ? global_symbols_link( library, descriptors, required ) : 0;
if ( !library_handle && ( flags & DYNAMIC_LINK_LOAD ) )
library_handle = dynamic_load( library, descriptors, required );
if ( !library_handle && ( flags & DYNAMIC_LINK_WEAK ) )
return weak_symbol_link( descriptors, required );
save_library_handle( library_handle, handle );
return true;
}
#endif /*__TBB_WIN8UI_SUPPORT*/
#else /* __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED */
bool dynamic_link( const char*, const dynamic_link_descriptor*, size_t, dynamic_link_handle *handle, int ) {
if ( handle )
*handle=0;
return false;
}
void dynamic_unlink( dynamic_link_handle ) {
}
void dynamic_unlink_all() {
}
#endif /* __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED */
CLOSE_INTERNAL_NAMESPACE

View File

@@ -0,0 +1,121 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_dynamic_link
#define __TBB_dynamic_link
// Support for dynamic loading entry points from other shared libraries.
#include "tbb/tbb_stddef.h"
#ifdef LIBRARY_ASSERT
#undef __TBB_ASSERT
#define __TBB_ASSERT(x,y) LIBRARY_ASSERT(x,y)
#else
#define LIBRARY_ASSERT(x,y) __TBB_ASSERT_EX(x,y)
#endif /* !LIBRARY_ASSERT */
/** By default, symbols declared and defined here go into namespace tbb::internal.
To put them in other namespace, define macros OPEN_INTERNAL_NAMESPACE
and CLOSE_INTERNAL_NAMESPACE to override the following default definitions. **/
#ifndef OPEN_INTERNAL_NAMESPACE
#define OPEN_INTERNAL_NAMESPACE namespace tbb { namespace internal {
#define CLOSE_INTERNAL_NAMESPACE }}
#endif /* OPEN_INTERNAL_NAMESPACE */
#include <stddef.h>
#if _WIN32
#include "tbb/machine/windows_api.h"
#endif /* _WIN32 */
OPEN_INTERNAL_NAMESPACE
//! Type definition for a pointer to a void somefunc(void)
typedef void (*pointer_to_handler)();
//! The helper to construct dynamic_link_descriptor structure
// Double cast through the void* in DLD macro is necessary to
// prevent warnings from some compilers (g++ 4.1)
#if __TBB_WEAK_SYMBOLS_PRESENT
#define DLD(s,h) {#s, (pointer_to_handler*)(void*)(&h), (pointer_to_handler)&s}
#else
#define DLD(s,h) {#s, (pointer_to_handler*)(void*)(&h)}
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
//! Association between a handler name and location of pointer to it.
struct dynamic_link_descriptor {
//! Name of the handler
const char* name;
//! Pointer to the handler
pointer_to_handler* handler;
#if __TBB_WEAK_SYMBOLS_PRESENT
//! Weak symbol
pointer_to_handler ptr;
#endif
};
#if _WIN32
typedef HMODULE dynamic_link_handle;
#else
typedef void* dynamic_link_handle;
#endif /* _WIN32 */
const int DYNAMIC_LINK_GLOBAL = 0x01;
const int DYNAMIC_LINK_LOAD = 0x02;
const int DYNAMIC_LINK_WEAK = 0x04;
const int DYNAMIC_LINK_ALL = DYNAMIC_LINK_GLOBAL | DYNAMIC_LINK_LOAD | DYNAMIC_LINK_WEAK;
//! Fill in dynamically linked handlers.
/** 'library' is the name of the requested library. It should not contain a full
path since dynamic_link adds the full path (from which the runtime itself
was loaded) to the library name.
'required' is the number of the initial entries in the array descriptors[]
that have to be found in order for the call to succeed. If the library and
all the required handlers are found, then the corresponding handler
pointers are set, and the return value is true. Otherwise the original
array of descriptors is left untouched and the return value is false.
'required' is limited by 20 (exceeding of this value will result in failure
to load the symbols and the return value will be false).
'handle' is the handle of the library if it is loaded. Otherwise it is left
untouched.
'flags' is the set of DYNAMIC_LINK_* flags. Each of the DYNAMIC_LINK_* flags
allows its corresponding linking stage.
**/
bool dynamic_link( const char* library,
const dynamic_link_descriptor descriptors[],
size_t required,
dynamic_link_handle* handle = 0,
int flags = DYNAMIC_LINK_ALL );
void dynamic_unlink( dynamic_link_handle handle );
void dynamic_unlink_all();
enum dynamic_link_error_t {
dl_success = 0,
dl_lib_not_found, // char const * lib, dlerr_t err
dl_sym_not_found, // char const * sym, dlerr_t err
// Note: dlerr_t depends on OS: it is char const * on Linux* and OS X*, int on Windows*.
dl_sys_fail, // char const * func, int err
dl_buff_too_small // none
}; // dynamic_link_error_t
CLOSE_INTERNAL_NAMESPACE
#endif /* __TBB_dynamic_link */

View File

@@ -0,0 +1,346 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include <stdio.h>
#include <stdlib.h>
#include "governor.h"
#include "tbb_main.h"
#include "scheduler.h"
#include "market.h"
#include "arena.h"
#include "tbb/task_scheduler_init.h"
#include "dynamic_link.h"
namespace tbb {
namespace internal {
//------------------------------------------------------------------------
// governor
//------------------------------------------------------------------------
#if __TBB_SURVIVE_THREAD_SWITCH
// Support for interoperability with Intel(R) Cilk(TM) Plus.
#if _WIN32
#define CILKLIB_NAME "cilkrts20.dll"
#else
#define CILKLIB_NAME "libcilkrts.so"
#endif
//! Handler for interoperation with cilkrts library.
static __cilk_tbb_retcode (*watch_stack_handler)(struct __cilk_tbb_unwatch_thunk* u,
struct __cilk_tbb_stack_op_thunk o);
//! Table describing how to link the handlers.
static const dynamic_link_descriptor CilkLinkTable[] = {
{ "__cilkrts_watch_stack", (pointer_to_handler*)(void*)(&watch_stack_handler) }
};
static atomic<do_once_state> cilkrts_load_state;
bool initialize_cilk_interop() {
// Pinning can fail. This is a normal situation, and means that the current
// thread does not use cilkrts and consequently does not need interop.
return dynamic_link( CILKLIB_NAME, CilkLinkTable, 1, /*handle=*/0, DYNAMIC_LINK_GLOBAL );
}
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
namespace rml {
tbb_server* make_private_server( tbb_client& client );
}
void governor::acquire_resources () {
#if USE_PTHREAD
int status = theTLS.create(auto_terminate);
#else
int status = theTLS.create();
#endif
if( status )
handle_perror(status, "TBB failed to initialize task scheduler TLS\n");
is_speculation_enabled = cpu_has_speculation();
}
void governor::release_resources () {
theRMLServerFactory.close();
#if TBB_USE_ASSERT
if( __TBB_InitOnce::initialization_done() && theTLS.get() )
runtime_warning( "TBB is unloaded while tbb::task_scheduler_init object is alive?" );
#endif
int status = theTLS.destroy();
if( status )
runtime_warning("failed to destroy task scheduler TLS: %s", strerror(status));
dynamic_unlink_all();
}
rml::tbb_server* governor::create_rml_server ( rml::tbb_client& client ) {
rml::tbb_server* server = NULL;
if( !UsePrivateRML ) {
::rml::factory::status_type status = theRMLServerFactory.make_server( server, client );
if( status != ::rml::factory::st_success ) {
UsePrivateRML = true;
runtime_warning( "rml::tbb_factory::make_server failed with status %x, falling back on private rml", status );
}
}
if ( !server ) {
__TBB_ASSERT( UsePrivateRML, NULL );
server = rml::make_private_server( client );
}
__TBB_ASSERT( server, "Failed to create RML server" );
return server;
}
void governor::sign_on(generic_scheduler* s) {
__TBB_ASSERT( !theTLS.get(), NULL );
theTLS.set(s);
#if __TBB_SURVIVE_THREAD_SWITCH
if( watch_stack_handler ) {
__cilk_tbb_stack_op_thunk o;
o.routine = &stack_op_handler;
o.data = s;
if( (*watch_stack_handler)(&s->my_cilk_unwatch_thunk, o) ) {
// Failed to register with cilkrts, make sure we are clean
s->my_cilk_unwatch_thunk.routine = NULL;
}
#if TBB_USE_ASSERT
else
s->my_cilk_state = generic_scheduler::cs_running;
#endif /* TBB_USE_ASSERT */
}
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
}
void governor::sign_off(generic_scheduler* s) {
suppress_unused_warning(s);
__TBB_ASSERT( theTLS.get()==s, "attempt to unregister a wrong scheduler instance" );
theTLS.set(NULL);
#if __TBB_SURVIVE_THREAD_SWITCH
__cilk_tbb_unwatch_thunk &ut = s->my_cilk_unwatch_thunk;
if ( ut.routine )
(*ut.routine)(ut.data);
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
}
void governor::setBlockingTerminate(const task_scheduler_init *tsi) {
__TBB_ASSERT(!IsBlockingTerminationInProgress, "It's impossible to create task_scheduler_init while blocking termination is in progress.");
if (BlockingTSI)
throw_exception(eid_blocking_sch_init);
BlockingTSI = tsi;
}
generic_scheduler* governor::init_scheduler( unsigned num_threads, stack_size_type stack_size, bool auto_init ) {
if( !__TBB_InitOnce::initialization_done() )
DoOneTimeInitializations();
generic_scheduler* s = theTLS.get();
if( s ) {
s->my_ref_count += 1;
return s;
}
#if __TBB_SURVIVE_THREAD_SWITCH
atomic_do_once( &initialize_cilk_interop, cilkrts_load_state );
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
if( (int)num_threads == task_scheduler_init::automatic )
num_threads = default_num_threads();
s = generic_scheduler::create_master(
market::create_arena( num_threads - 1, stack_size ? stack_size : ThreadStackSize ) );
__TBB_ASSERT(s, "Somehow a local scheduler creation for a master thread failed");
s->my_auto_initialized = auto_init;
return s;
}
void governor::terminate_scheduler( generic_scheduler* s, const task_scheduler_init* tsi_ptr ) {
__TBB_ASSERT( s == theTLS.get(), "Attempt to terminate non-local scheduler instance" );
if (--(s->my_ref_count)) {
if (BlockingTSI && BlockingTSI==tsi_ptr) {
// can't throw exception, because this is on dtor's call chain
fprintf(stderr, "Attempt to terminate nested scheduler in blocking mode\n");
exit(1);
}
} else {
#if TBB_USE_ASSERT
if (BlockingTSI) {
__TBB_ASSERT( BlockingTSI == tsi_ptr, "For blocking termination last terminate_scheduler must be blocking." );
IsBlockingTerminationInProgress = true;
}
#endif
s->cleanup_master();
BlockingTSI = NULL;
#if TBB_USE_ASSERT
IsBlockingTerminationInProgress = false;
#endif
}
}
void governor::auto_terminate(void* arg){
generic_scheduler* s = static_cast<generic_scheduler*>(arg);
if( s && s->my_auto_initialized ) {
if( !--(s->my_ref_count) ) {
__TBB_ASSERT( !BlockingTSI, "Blocking auto-terminate is not supported." );
// If the TLS slot is already cleared by OS or underlying concurrency
// runtime, restore its value.
if ( !theTLS.get() )
theTLS.set(s);
else __TBB_ASSERT( s == theTLS.get(), NULL );
s->cleanup_master();
__TBB_ASSERT( !theTLS.get(), "cleanup_master has not cleared its TLS slot" );
}
}
}
void governor::print_version_info () {
if ( UsePrivateRML )
PrintExtraVersionInfo( "RML", "private" );
else {
PrintExtraVersionInfo( "RML", "shared" );
theRMLServerFactory.call_with_server_info( PrintRMLVersionInfo, (void*)"" );
}
#if __TBB_SURVIVE_THREAD_SWITCH
if( watch_stack_handler )
PrintExtraVersionInfo( "CILK", CILKLIB_NAME );
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
}
void governor::initialize_rml_factory () {
::rml::factory::status_type res = theRMLServerFactory.open();
UsePrivateRML = res != ::rml::factory::st_success;
}
#if __TBB_SURVIVE_THREAD_SWITCH
__cilk_tbb_retcode governor::stack_op_handler( __cilk_tbb_stack_op op, void* data ) {
__TBB_ASSERT(data,NULL);
generic_scheduler* s = static_cast<generic_scheduler*>(data);
#if TBB_USE_ASSERT
void* current = theTLS.get();
#if _WIN32||_WIN64
uintptr_t thread_id = GetCurrentThreadId();
#else
uintptr_t thread_id = uintptr_t(pthread_self());
#endif
#endif /* TBB_USE_ASSERT */
switch( op ) {
default:
__TBB_ASSERT( 0, "invalid op" );
case CILK_TBB_STACK_ADOPT: {
__TBB_ASSERT( !current && s->my_cilk_state==generic_scheduler::cs_limbo ||
current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid adoption" );
#if TBB_USE_ASSERT
if( current==s )
runtime_warning( "redundant adoption of %p by thread %p\n", s, (void*)thread_id );
s->my_cilk_state = generic_scheduler::cs_running;
#endif /* TBB_USE_ASSERT */
theTLS.set(s);
break;
}
case CILK_TBB_STACK_ORPHAN: {
__TBB_ASSERT( current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid orphaning" );
#if TBB_USE_ASSERT
s->my_cilk_state = generic_scheduler::cs_limbo;
#endif /* TBB_USE_ASSERT */
theTLS.set(NULL);
break;
}
case CILK_TBB_STACK_RELEASE: {
__TBB_ASSERT( !current && s->my_cilk_state==generic_scheduler::cs_limbo ||
current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid release" );
#if TBB_USE_ASSERT
s->my_cilk_state = generic_scheduler::cs_freed;
#endif /* TBB_USE_ASSERT */
s->my_cilk_unwatch_thunk.routine = NULL;
auto_terminate( s );
}
}
return 0;
}
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
} // namespace internal
//------------------------------------------------------------------------
// task_scheduler_init
//------------------------------------------------------------------------
using namespace internal;
/** Left out-of-line for the sake of the backward binary compatibility **/
void task_scheduler_init::initialize( int number_of_threads ) {
initialize( number_of_threads, 0 );
}
void task_scheduler_init::initialize( int number_of_threads, stack_size_type thread_stack_size ) {
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
uintptr_t new_mode = thread_stack_size & propagation_mode_mask;
#endif
thread_stack_size &= ~(stack_size_type)propagation_mode_mask;
if( number_of_threads!=deferred ) {
bool blocking_terminate = false;
if (my_scheduler == (scheduler*)wait_workers_in_terminate_flag) {
blocking_terminate = true;
my_scheduler = NULL;
}
__TBB_ASSERT( !my_scheduler, "task_scheduler_init already initialized" );
__TBB_ASSERT( number_of_threads==-1 || number_of_threads>=1,
"number_of_threads for task_scheduler_init must be -1 or positive" );
if (blocking_terminate)
governor::setBlockingTerminate(this);
internal::generic_scheduler *s = governor::init_scheduler( number_of_threads, thread_stack_size, /*auto_init=*/false );
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
if ( s->master_outermost_level() ) {
uintptr_t &vt = s->default_context()->my_version_and_traits;
uintptr_t prev_mode = vt & task_group_context::exact_exception ? propagation_mode_exact : 0;
vt = new_mode & propagation_mode_exact ? vt | task_group_context::exact_exception
: new_mode & propagation_mode_captured ? vt & ~task_group_context::exact_exception : vt;
// Use least significant bit of the scheduler pointer to store previous mode.
// This is necessary when components compiled with different compilers and/or
// TBB versions initialize the
my_scheduler = static_cast<scheduler*>((generic_scheduler*)((uintptr_t)s | prev_mode));
}
else
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
my_scheduler = s;
} else {
__TBB_ASSERT( !thread_stack_size, "deferred initialization ignores stack size setting" );
}
}
void task_scheduler_init::terminate() {
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
uintptr_t prev_mode = (uintptr_t)my_scheduler & propagation_mode_exact;
my_scheduler = (scheduler*)((uintptr_t)my_scheduler & ~(uintptr_t)propagation_mode_exact);
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
generic_scheduler* s = static_cast<generic_scheduler*>(my_scheduler);
my_scheduler = NULL;
__TBB_ASSERT( s, "task_scheduler_init::terminate without corresponding task_scheduler_init::initialize()");
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
if ( s->master_outermost_level() ) {
uintptr_t &vt = s->default_context()->my_version_and_traits;
vt = prev_mode & propagation_mode_exact ? vt | task_group_context::exact_exception
: vt & ~task_group_context::exact_exception;
}
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
governor::terminate_scheduler(s, this);
}
int task_scheduler_init::default_num_threads() {
return governor::default_num_threads();
}
} // namespace tbb

View File

@@ -0,0 +1,146 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_governor_H
#define _TBB_governor_H
#include "tbb/task_scheduler_init.h"
#include "../rml/include/rml_tbb.h"
#include "tbb_misc.h" // for AvailableHwConcurrency and ThreadStackSize
#include "tls.h"
#if __TBB_SURVIVE_THREAD_SWITCH
#include "cilk-tbb-interop.h"
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
namespace tbb {
namespace internal {
class market;
class generic_scheduler;
class __TBB_InitOnce;
//------------------------------------------------------------------------
// Class governor
//------------------------------------------------------------------------
//! The class handles access to the single instance of market, and to TLS to keep scheduler instances.
/** It also supports automatic on-demand initialization of the TBB scheduler.
The class contains only static data members and methods.*/
class governor {
friend class __TBB_InitOnce;
friend class market;
//! TLS for scheduler instances associated with individual threads
static basic_tls<generic_scheduler*> theTLS;
//! Caches the maximal level of parallelism supported by the hardware
static unsigned DefaultNumberOfThreads;
static rml::tbb_factory theRMLServerFactory;
static bool UsePrivateRML;
//! Instance of task_scheduler_init that requested blocking termination.
static const task_scheduler_init *BlockingTSI;
#if TBB_USE_ASSERT
static bool IsBlockingTerminationInProgress;
#endif
static bool is_speculation_enabled;
//! Create key for thread-local storage and initialize RML.
static void acquire_resources ();
//! Destroy the thread-local storage key and deinitialize RML.
static void release_resources ();
static rml::tbb_server* create_rml_server ( rml::tbb_client& );
//! The internal routine to undo automatic initialization.
/** The signature is written with void* so that the routine
can be the destructor argument to pthread_key_create. */
static void auto_terminate(void* scheduler);
public:
static unsigned default_num_threads () {
// No memory fence required, because at worst each invoking thread calls AvailableHwConcurrency once.
return DefaultNumberOfThreads ? DefaultNumberOfThreads :
DefaultNumberOfThreads = AvailableHwConcurrency();
}
//! Processes scheduler initialization request (possibly nested) in a master thread
/** If necessary creates new instance of arena and/or local scheduler.
The auto_init argument specifies if the call is due to automatic initialization. **/
static generic_scheduler* init_scheduler( unsigned num_threads, stack_size_type stack_size, bool auto_init = false );
//! Processes scheduler termination request (possibly nested) in a master thread
static void terminate_scheduler( generic_scheduler* s, const task_scheduler_init *tsi_ptr );
//! Register TBB scheduler instance in thread-local storage.
static void sign_on(generic_scheduler* s);
//! Unregister TBB scheduler instance from thread-local storage.
static void sign_off(generic_scheduler* s);
//! Used to check validity of the local scheduler TLS contents.
static bool is_set ( generic_scheduler* s ) { return theTLS.get() == s; }
//! Temporarily set TLS slot to the given scheduler
static void assume_scheduler( generic_scheduler* s ) { theTLS.set( s ); }
//! Obtain the thread-local instance of the TBB scheduler.
/** If the scheduler has not been initialized yet, initialization is done automatically.
Note that auto-initialized scheduler instance is destroyed only when its thread terminates. **/
static generic_scheduler* local_scheduler () {
generic_scheduler* s = theTLS.get();
return s ? s : init_scheduler( (unsigned)task_scheduler_init::automatic, 0, true );
}
static generic_scheduler* local_scheduler_if_initialized () {
return theTLS.get();
}
//! Undo automatic initialization if necessary; call when a thread exits.
static void terminate_auto_initialized_scheduler() {
auto_terminate( theTLS.get() );
}
static void print_version_info ();
static void initialize_rml_factory ();
static bool needsWaitWorkers () { return BlockingTSI!=NULL; }
//! Must be called before init_scheduler
static void setBlockingTerminate(const task_scheduler_init *tsi);
#if __TBB_SURVIVE_THREAD_SWITCH
static __cilk_tbb_retcode stack_op_handler( __cilk_tbb_stack_op op, void* );
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
static bool speculation_enabled() { return is_speculation_enabled; }
}; // class governor
} // namespace internal
} // namespace tbb
#endif /* _TBB_governor_H */

View File

@@ -0,0 +1,188 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
.686
.model flat,c
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchadd1
__TBB_machine_fetchadd1:
mov edx,4[esp]
mov eax,8[esp]
lock xadd [edx],al
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchstore1
__TBB_machine_fetchstore1:
mov edx,4[esp]
mov eax,8[esp]
lock xchg [edx],al
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_cmpswp1
__TBB_machine_cmpswp1:
mov edx,4[esp]
mov ecx,8[esp]
mov eax,12[esp]
lock cmpxchg [edx],cl
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchadd2
__TBB_machine_fetchadd2:
mov edx,4[esp]
mov eax,8[esp]
lock xadd [edx],ax
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchstore2
__TBB_machine_fetchstore2:
mov edx,4[esp]
mov eax,8[esp]
lock xchg [edx],ax
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_cmpswp2
__TBB_machine_cmpswp2:
mov edx,4[esp]
mov ecx,8[esp]
mov eax,12[esp]
lock cmpxchg [edx],cx
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchadd4
__TBB_machine_fetchadd4:
mov edx,4[esp]
mov eax,8[esp]
lock xadd [edx],eax
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchstore4
__TBB_machine_fetchstore4:
mov edx,4[esp]
mov eax,8[esp]
lock xchg [edx],eax
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_cmpswp4
__TBB_machine_cmpswp4:
mov edx,4[esp]
mov ecx,8[esp]
mov eax,12[esp]
lock cmpxchg [edx],ecx
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchadd8
__TBB_machine_fetchadd8:
push ebx
push edi
mov edi,12[esp]
mov eax,[edi]
mov edx,4[edi]
__TBB_machine_fetchadd8_loop:
mov ebx,16[esp]
mov ecx,20[esp]
add ebx,eax
adc ecx,edx
lock cmpxchg8b qword ptr [edi]
jnz __TBB_machine_fetchadd8_loop
pop edi
pop ebx
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_fetchstore8
__TBB_machine_fetchstore8:
push ebx
push edi
mov edi,12[esp]
mov ebx,16[esp]
mov ecx,20[esp]
mov eax,[edi]
mov edx,4[edi]
__TBB_machine_fetchstore8_loop:
lock cmpxchg8b qword ptr [edi]
jnz __TBB_machine_fetchstore8_loop
pop edi
pop ebx
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_cmpswp8
__TBB_machine_cmpswp8:
push ebx
push edi
mov edi,12[esp]
mov ebx,16[esp]
mov ecx,20[esp]
mov eax,24[esp]
mov edx,28[esp]
lock cmpxchg8b qword ptr [edi]
pop edi
pop ebx
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_load8
__TBB_machine_Load8:
; If location is on stack, compiler may have failed to align it correctly, so we do dynamic check.
mov ecx,4[esp]
test ecx,7
jne load_slow
; Load within a cache line
sub esp,12
fild qword ptr [ecx]
fistp qword ptr [esp]
mov eax,[esp]
mov edx,4[esp]
add esp,12
ret
load_slow:
; Load is misaligned. Use cmpxchg8b.
push ebx
push edi
mov edi,ecx
xor eax,eax
xor ebx,ebx
xor ecx,ecx
xor edx,edx
lock cmpxchg8b qword ptr [edi]
pop edi
pop ebx
ret
EXTRN __TBB_machine_store8_slow:PROC
.code
ALIGN 4
PUBLIC c __TBB_machine_store8
__TBB_machine_Store8:
; If location is on stack, compiler may have failed to align it correctly, so we do dynamic check.
mov ecx,4[esp]
test ecx,7
jne __TBB_machine_store8_slow ;; tail call to tbb_misc.cpp
fild qword ptr 8[esp]
fistp qword ptr [ecx]
ret
end

View File

@@ -0,0 +1,80 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
.686
.model flat,c
.code
ALIGN 4
PUBLIC c __TBB_machine_try_lock_elided
__TBB_machine_try_lock_elided:
mov ecx, 4[esp]
xor eax, eax
mov al, 1
BYTE 0F2H
xchg al, byte ptr [ecx]
xor al, 1
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_unlock_elided
__TBB_machine_unlock_elided:
mov ecx, 4[esp]
BYTE 0F3H
mov byte ptr [ecx], 0
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_begin_transaction
__TBB_machine_begin_transaction:
mov eax, -1
BYTE 0C7H
BYTE 0F8H
BYTE 000H
BYTE 000H
BYTE 000H
BYTE 000H
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_end_transaction
__TBB_machine_end_transaction:
BYTE 00FH
BYTE 001H
BYTE 0D5H
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_transaction_conflict_abort
__TBB_machine_transaction_conflict_abort:
BYTE 0C6H
BYTE 0F8H
BYTE 0FFH ; 12.4.5 Abort argument: lock not free when tested
ret
.code
ALIGN 4
PUBLIC c __TBB_machine_is_in_transaction
__TBB_machine_is_in_transaction:
xor eax, eax
BYTE 00FH
BYTE 001H
BYTE 0D6H
JZ rset
MOV al,1
rset:
RET
end

View File

@@ -0,0 +1,38 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
; DO NOT EDIT - AUTOMATICALLY GENERATED FROM .s FILE
.686
.model flat,c
.code
ALIGN 4
PUBLIC c __TBB_machine_trylockbyte
__TBB_machine_trylockbyte:
mov edx,4[esp]
mov al,[edx]
mov cl,1
test al,1
jnz __TBB_machine_trylockbyte_contended
lock cmpxchg [edx],cl
jne __TBB_machine_trylockbyte_contended
mov eax,1
ret
__TBB_machine_trylockbyte_contended:
xor eax,eax
ret
end

View File

@@ -0,0 +1,670 @@
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
//
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
// you can redistribute it and/or modify it under the terms of the GNU General Public License
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
// See the GNU General Public License for more details. You should have received a copy of
// the GNU General Public License along with Threading Building Blocks; if not, write to the
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
//
// As a special exception, you may use this file as part of a free software library without
// restriction. Specifically, if other files instantiate templates or use macros or inline
// functions from this file, or you compile this file and link it with other files to produce
// an executable, this file does not by itself cause the resulting executable to be covered
// by the GNU General Public License. This exception does not however invalidate any other
// reasons why the executable file might be covered by the GNU General Public License.
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchadd1__TBB_full_fence#
.global __TBB_machine_fetchadd1__TBB_full_fence#
__TBB_machine_fetchadd1__TBB_full_fence:
{
mf
br __TBB_machine_fetchadd1acquire
}
.endp __TBB_machine_fetchadd1__TBB_full_fence#
.proc __TBB_machine_fetchadd1acquire#
.global __TBB_machine_fetchadd1acquire#
__TBB_machine_fetchadd1acquire:
ld1 r9=[r32]
;;
Retry_1acquire:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg1.acq r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_1acquire
br.ret.sptk.many b0
# 49 "<stdin>"
.endp __TBB_machine_fetchadd1acquire#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore1__TBB_full_fence#
.global __TBB_machine_fetchstore1__TBB_full_fence#
__TBB_machine_fetchstore1__TBB_full_fence:
mf
;;
xchg1 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore1__TBB_full_fence#
.proc __TBB_machine_fetchstore1acquire#
.global __TBB_machine_fetchstore1acquire#
__TBB_machine_fetchstore1acquire:
xchg1 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore1acquire#
# 88 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_cmpswp1__TBB_full_fence#
.global __TBB_machine_cmpswp1__TBB_full_fence#
__TBB_machine_cmpswp1__TBB_full_fence:
{
mf
br __TBB_machine_cmpswp1acquire
}
.endp __TBB_machine_cmpswp1__TBB_full_fence#
.proc __TBB_machine_cmpswp1acquire#
.global __TBB_machine_cmpswp1acquire#
__TBB_machine_cmpswp1acquire:
zxt1 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg1.acq r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp1acquire#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchadd2__TBB_full_fence#
.global __TBB_machine_fetchadd2__TBB_full_fence#
__TBB_machine_fetchadd2__TBB_full_fence:
{
mf
br __TBB_machine_fetchadd2acquire
}
.endp __TBB_machine_fetchadd2__TBB_full_fence#
.proc __TBB_machine_fetchadd2acquire#
.global __TBB_machine_fetchadd2acquire#
__TBB_machine_fetchadd2acquire:
ld2 r9=[r32]
;;
Retry_2acquire:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg2.acq r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_2acquire
br.ret.sptk.many b0
# 49 "<stdin>"
.endp __TBB_machine_fetchadd2acquire#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore2__TBB_full_fence#
.global __TBB_machine_fetchstore2__TBB_full_fence#
__TBB_machine_fetchstore2__TBB_full_fence:
mf
;;
xchg2 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore2__TBB_full_fence#
.proc __TBB_machine_fetchstore2acquire#
.global __TBB_machine_fetchstore2acquire#
__TBB_machine_fetchstore2acquire:
xchg2 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore2acquire#
# 88 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_cmpswp2__TBB_full_fence#
.global __TBB_machine_cmpswp2__TBB_full_fence#
__TBB_machine_cmpswp2__TBB_full_fence:
{
mf
br __TBB_machine_cmpswp2acquire
}
.endp __TBB_machine_cmpswp2__TBB_full_fence#
.proc __TBB_machine_cmpswp2acquire#
.global __TBB_machine_cmpswp2acquire#
__TBB_machine_cmpswp2acquire:
zxt2 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg2.acq r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp2acquire#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchadd4__TBB_full_fence#
.global __TBB_machine_fetchadd4__TBB_full_fence#
__TBB_machine_fetchadd4__TBB_full_fence:
{
mf
br __TBB_machine_fetchadd4acquire
}
.endp __TBB_machine_fetchadd4__TBB_full_fence#
.proc __TBB_machine_fetchadd4acquire#
.global __TBB_machine_fetchadd4acquire#
__TBB_machine_fetchadd4acquire:
cmp.eq p6,p0=1,r33
cmp.eq p8,p0=-1,r33
(p6) br.cond.dptk Inc_4acquire
(p8) br.cond.dpnt Dec_4acquire
;;
ld4 r9=[r32]
;;
Retry_4acquire:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg4.acq r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_4acquire
br.ret.sptk.many b0
Inc_4acquire:
fetchadd4.acq r8=[r32],1
br.ret.sptk.many b0
Dec_4acquire:
fetchadd4.acq r8=[r32],-1
br.ret.sptk.many b0
.endp __TBB_machine_fetchadd4acquire#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore4__TBB_full_fence#
.global __TBB_machine_fetchstore4__TBB_full_fence#
__TBB_machine_fetchstore4__TBB_full_fence:
mf
;;
xchg4 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore4__TBB_full_fence#
.proc __TBB_machine_fetchstore4acquire#
.global __TBB_machine_fetchstore4acquire#
__TBB_machine_fetchstore4acquire:
xchg4 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore4acquire#
# 88 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_cmpswp4__TBB_full_fence#
.global __TBB_machine_cmpswp4__TBB_full_fence#
__TBB_machine_cmpswp4__TBB_full_fence:
{
mf
br __TBB_machine_cmpswp4acquire
}
.endp __TBB_machine_cmpswp4__TBB_full_fence#
.proc __TBB_machine_cmpswp4acquire#
.global __TBB_machine_cmpswp4acquire#
__TBB_machine_cmpswp4acquire:
zxt4 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg4.acq r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp4acquire#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchadd8__TBB_full_fence#
.global __TBB_machine_fetchadd8__TBB_full_fence#
__TBB_machine_fetchadd8__TBB_full_fence:
{
mf
br __TBB_machine_fetchadd8acquire
}
.endp __TBB_machine_fetchadd8__TBB_full_fence#
.proc __TBB_machine_fetchadd8acquire#
.global __TBB_machine_fetchadd8acquire#
__TBB_machine_fetchadd8acquire:
cmp.eq p6,p0=1,r33
cmp.eq p8,p0=-1,r33
(p6) br.cond.dptk Inc_8acquire
(p8) br.cond.dpnt Dec_8acquire
;;
ld8 r9=[r32]
;;
Retry_8acquire:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg8.acq r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_8acquire
br.ret.sptk.many b0
Inc_8acquire:
fetchadd8.acq r8=[r32],1
br.ret.sptk.many b0
Dec_8acquire:
fetchadd8.acq r8=[r32],-1
br.ret.sptk.many b0
.endp __TBB_machine_fetchadd8acquire#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore8__TBB_full_fence#
.global __TBB_machine_fetchstore8__TBB_full_fence#
__TBB_machine_fetchstore8__TBB_full_fence:
mf
;;
xchg8 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore8__TBB_full_fence#
.proc __TBB_machine_fetchstore8acquire#
.global __TBB_machine_fetchstore8acquire#
__TBB_machine_fetchstore8acquire:
xchg8 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore8acquire#
# 88 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_cmpswp8__TBB_full_fence#
.global __TBB_machine_cmpswp8__TBB_full_fence#
__TBB_machine_cmpswp8__TBB_full_fence:
{
mf
br __TBB_machine_cmpswp8acquire
}
.endp __TBB_machine_cmpswp8__TBB_full_fence#
.proc __TBB_machine_cmpswp8acquire#
.global __TBB_machine_cmpswp8acquire#
__TBB_machine_cmpswp8acquire:
mov ar.ccv=r34
;;
cmpxchg8.acq r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp8acquire#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
# 19 "<stdin>"
.proc __TBB_machine_fetchadd1release#
.global __TBB_machine_fetchadd1release#
__TBB_machine_fetchadd1release:
ld1 r9=[r32]
;;
Retry_1release:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg1.rel r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_1release
br.ret.sptk.many b0
# 49 "<stdin>"
.endp __TBB_machine_fetchadd1release#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore1release#
.global __TBB_machine_fetchstore1release#
__TBB_machine_fetchstore1release:
mf
;;
xchg1 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore1release#
# 88 "<stdin>"
.section .text
.align 16
# 101 "<stdin>"
.proc __TBB_machine_cmpswp1release#
.global __TBB_machine_cmpswp1release#
__TBB_machine_cmpswp1release:
zxt1 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg1.rel r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp1release#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
# 19 "<stdin>"
.proc __TBB_machine_fetchadd2release#
.global __TBB_machine_fetchadd2release#
__TBB_machine_fetchadd2release:
ld2 r9=[r32]
;;
Retry_2release:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg2.rel r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_2release
br.ret.sptk.many b0
# 49 "<stdin>"
.endp __TBB_machine_fetchadd2release#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore2release#
.global __TBB_machine_fetchstore2release#
__TBB_machine_fetchstore2release:
mf
;;
xchg2 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore2release#
# 88 "<stdin>"
.section .text
.align 16
# 101 "<stdin>"
.proc __TBB_machine_cmpswp2release#
.global __TBB_machine_cmpswp2release#
__TBB_machine_cmpswp2release:
zxt2 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg2.rel r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp2release#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
# 19 "<stdin>"
.proc __TBB_machine_fetchadd4release#
.global __TBB_machine_fetchadd4release#
__TBB_machine_fetchadd4release:
cmp.eq p6,p0=1,r33
cmp.eq p8,p0=-1,r33
(p6) br.cond.dptk Inc_4release
(p8) br.cond.dpnt Dec_4release
;;
ld4 r9=[r32]
;;
Retry_4release:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg4.rel r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_4release
br.ret.sptk.many b0
Inc_4release:
fetchadd4.rel r8=[r32],1
br.ret.sptk.many b0
Dec_4release:
fetchadd4.rel r8=[r32],-1
br.ret.sptk.many b0
.endp __TBB_machine_fetchadd4release#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore4release#
.global __TBB_machine_fetchstore4release#
__TBB_machine_fetchstore4release:
mf
;;
xchg4 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore4release#
# 88 "<stdin>"
.section .text
.align 16
# 101 "<stdin>"
.proc __TBB_machine_cmpswp4release#
.global __TBB_machine_cmpswp4release#
__TBB_machine_cmpswp4release:
zxt4 r34=r34
;;
mov ar.ccv=r34
;;
cmpxchg4.rel r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp4release#
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "<stdin>"
.section .text
.align 16
# 19 "<stdin>"
.proc __TBB_machine_fetchadd8release#
.global __TBB_machine_fetchadd8release#
__TBB_machine_fetchadd8release:
cmp.eq p6,p0=1,r33
cmp.eq p8,p0=-1,r33
(p6) br.cond.dptk Inc_8release
(p8) br.cond.dpnt Dec_8release
;;
ld8 r9=[r32]
;;
Retry_8release:
mov ar.ccv=r9
mov r8=r9;
add r10=r9,r33
;;
cmpxchg8.rel r9=[r32],r10,ar.ccv
;;
cmp.ne p7,p0=r8,r9
(p7) br.cond.dpnt Retry_8release
br.ret.sptk.many b0
Inc_8release:
fetchadd8.rel r8=[r32],1
br.ret.sptk.many b0
Dec_8release:
fetchadd8.rel r8=[r32],-1
br.ret.sptk.many b0
.endp __TBB_machine_fetchadd8release#
# 62 "<stdin>"
.section .text
.align 16
.proc __TBB_machine_fetchstore8release#
.global __TBB_machine_fetchstore8release#
__TBB_machine_fetchstore8release:
mf
;;
xchg8 r8=[r32],r33
br.ret.sptk.many b0
.endp __TBB_machine_fetchstore8release#
# 88 "<stdin>"
.section .text
.align 16
# 101 "<stdin>"
.proc __TBB_machine_cmpswp8release#
.global __TBB_machine_cmpswp8release#
__TBB_machine_cmpswp8release:
mov ar.ccv=r34
;;
cmpxchg8.rel r8=[r32],r33,ar.ccv
br.ret.sptk.many b0
.endp __TBB_machine_cmpswp8release#

View File

@@ -0,0 +1,99 @@
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
//
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
// you can redistribute it and/or modify it under the terms of the GNU General Public License
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
// See the GNU General Public License for more details. You should have received a copy of
// the GNU General Public License along with Threading Building Blocks; if not, write to the
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
//
// As a special exception, you may use this file as part of a free software library without
// restriction. Specifically, if other files instantiate templates or use macros or inline
// functions from this file, or you compile this file and link it with other files to produce
// an executable, this file does not by itself cause the resulting executable to be covered
// by the GNU General Public License. This exception does not however invalidate any other
// reasons why the executable file might be covered by the GNU General Public License.
// RSE backing store pointer retrieval
.section .text
.align 16
.proc __TBB_get_bsp#
.global __TBB_get_bsp#
__TBB_get_bsp:
mov r8=ar.bsp
br.ret.sptk.many b0
.endp __TBB_get_bsp#
.section .text
.align 16
.proc __TBB_machine_load8_relaxed#
.global __TBB_machine_load8_relaxed#
__TBB_machine_load8_relaxed:
ld8 r8=[r32]
br.ret.sptk.many b0
.endp __TBB_machine_load8_relaxed#
.section .text
.align 16
.proc __TBB_machine_store8_relaxed#
.global __TBB_machine_store8_relaxed#
__TBB_machine_store8_relaxed:
st8 [r32]=r33
br.ret.sptk.many b0
.endp __TBB_machine_store8_relaxed#
.section .text
.align 16
.proc __TBB_machine_load4_relaxed#
.global __TBB_machine_load4_relaxed#
__TBB_machine_load4_relaxed:
ld4 r8=[r32]
br.ret.sptk.many b0
.endp __TBB_machine_load4_relaxed#
.section .text
.align 16
.proc __TBB_machine_store4_relaxed#
.global __TBB_machine_store4_relaxed#
__TBB_machine_store4_relaxed:
st4 [r32]=r33
br.ret.sptk.many b0
.endp __TBB_machine_store4_relaxed#
.section .text
.align 16
.proc __TBB_machine_load2_relaxed#
.global __TBB_machine_load2_relaxed#
__TBB_machine_load2_relaxed:
ld2 r8=[r32]
br.ret.sptk.many b0
.endp __TBB_machine_load2_relaxed#
.section .text
.align 16
.proc __TBB_machine_store2_relaxed#
.global __TBB_machine_store2_relaxed#
__TBB_machine_store2_relaxed:
st2 [r32]=r33
br.ret.sptk.many b0
.endp __TBB_machine_store2_relaxed#
.section .text
.align 16
.proc __TBB_machine_load1_relaxed#
.global __TBB_machine_load1_relaxed#
__TBB_machine_load1_relaxed:
ld1 r8=[r32]
br.ret.sptk.many b0
.endp __TBB_machine_load1_relaxed#
.section .text
.align 16
.proc __TBB_machine_store1_relaxed#
.global __TBB_machine_store1_relaxed#
__TBB_machine_store1_relaxed:
st1 [r32]=r33
br.ret.sptk.many b0
.endp __TBB_machine_store1_relaxed#

View File

@@ -0,0 +1,46 @@
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
//
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
// you can redistribute it and/or modify it under the terms of the GNU General Public License
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
// See the GNU General Public License for more details. You should have received a copy of
// the GNU General Public License along with Threading Building Blocks; if not, write to the
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
//
// As a special exception, you may use this file as part of a free software library without
// restriction. Specifically, if other files instantiate templates or use macros or inline
// functions from this file, or you compile this file and link it with other files to produce
// an executable, this file does not by itself cause the resulting executable to be covered
// by the GNU General Public License. This exception does not however invalidate any other
// reasons why the executable file might be covered by the GNU General Public License.
// Support for class TinyLock
.section .text
.align 16
// unsigned int __TBB_machine_trylockbyte( byte& flag );
// r32 = address of flag
.proc __TBB_machine_trylockbyte#
.global __TBB_machine_trylockbyte#
ADDRESS_OF_FLAG=r32
RETCODE=r8
FLAG=r9
BUSY=r10
SCRATCH=r11
__TBB_machine_trylockbyte:
ld1.acq FLAG=[ADDRESS_OF_FLAG]
mov BUSY=1
mov RETCODE=0
;;
cmp.ne p6,p0=0,FLAG
mov ar.ccv=r0
(p6) br.ret.sptk.many b0
;;
cmpxchg1.acq SCRATCH=[ADDRESS_OF_FLAG],BUSY,ar.ccv // Try to acquire lock
;;
cmp.eq p6,p0=0,SCRATCH
;;
(p6) mov RETCODE=1
br.ret.sptk.many b0
.endp __TBB_machine_trylockbyte#

View File

@@ -0,0 +1,58 @@
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
//
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
// you can redistribute it and/or modify it under the terms of the GNU General Public License
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
// See the GNU General Public License for more details. You should have received a copy of
// the GNU General Public License along with Threading Building Blocks; if not, write to the
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
//
// As a special exception, you may use this file as part of a free software library without
// restriction. Specifically, if other files instantiate templates or use macros or inline
// functions from this file, or you compile this file and link it with other files to produce
// an executable, this file does not by itself cause the resulting executable to be covered
// by the GNU General Public License. This exception does not however invalidate any other
// reasons why the executable file might be covered by the GNU General Public License.
.section .text
.align 16
// unsigned long __TBB_machine_lg( unsigned long x );
// r32 = x
.proc __TBB_machine_lg#
.global __TBB_machine_lg#
__TBB_machine_lg:
shr r16=r32,1 // .x
;;
shr r17=r32,2 // ..x
or r32=r32,r16 // xx
;;
shr r16=r32,3 // ...xx
or r32=r32,r17 // xxx
;;
shr r17=r32,5 // .....xxx
or r32=r32,r16 // xxxxx
;;
shr r16=r32,8 // ........xxxxx
or r32=r32,r17 // xxxxxxxx
;;
shr r17=r32,13
or r32=r32,r16 // 13x
;;
shr r16=r32,21
or r32=r32,r17 // 21x
;;
shr r17=r32,34
or r32=r32,r16 // 34x
;;
shr r16=r32,55
or r32=r32,r17 // 55x
;;
or r32=r32,r16 // 64x
;;
popcnt r8=r32
;;
add r8=-1,r8
br.ret.sptk.many b0
.endp __TBB_machine_lg#

View File

@@ -0,0 +1,33 @@
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
//
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
// you can redistribute it and/or modify it under the terms of the GNU General Public License
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
// See the GNU General Public License for more details. You should have received a copy of
// the GNU General Public License along with Threading Building Blocks; if not, write to the
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
//
// As a special exception, you may use this file as part of a free software library without
// restriction. Specifically, if other files instantiate templates or use macros or inline
// functions from this file, or you compile this file and link it with other files to produce
// an executable, this file does not by itself cause the resulting executable to be covered
// by the GNU General Public License. This exception does not however invalidate any other
// reasons why the executable file might be covered by the GNU General Public License.
.section .text
.align 16
// void __TBB_machine_pause( long count );
// r32 = count
.proc __TBB_machine_pause#
.global __TBB_machine_pause#
count = r32
__TBB_machine_pause:
hint.m 0
add count=-1,count
;;
cmp.eq p6,p7=0,count
(p7) br.cond.dpnt __TBB_machine_pause
(p6) br.ret.sptk.many b0
.endp __TBB_machine_pause#

View File

@@ -0,0 +1,55 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include <stdint.h>
#include <sys/atomic_op.h>
/* This file must be compiled with gcc. The IBM compiler doesn't seem to
support inline assembly statements (October 2007). */
#ifdef __GNUC__
int32_t __TBB_machine_cas_32 (volatile void* ptr, int32_t value, int32_t comparand) {
__asm__ __volatile__ ("sync\n"); /* memory release operation */
compare_and_swap ((atomic_p) ptr, &comparand, value);
__asm__ __volatile__ ("isync\n"); /* memory acquire operation */
return comparand;
}
int64_t __TBB_machine_cas_64 (volatile void* ptr, int64_t value, int64_t comparand) {
__asm__ __volatile__ ("sync\n"); /* memory release operation */
compare_and_swaplp ((atomic_l) ptr, &comparand, value);
__asm__ __volatile__ ("isync\n"); /* memory acquire operation */
return comparand;
}
void __TBB_machine_flush () {
__asm__ __volatile__ ("sync\n");
}
void __TBB_machine_lwsync () {
__asm__ __volatile__ ("lwsync\n");
}
void __TBB_machine_isync () {
__asm__ __volatile__ ("isync\n");
}
#endif /* __GNUC__ */

View File

@@ -0,0 +1,72 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
; DO NOT EDIT - AUTOMATICALLY GENERATED FROM .s FILE
.code
ALIGN 8
PUBLIC __TBB_machine_fetchadd1
__TBB_machine_fetchadd1:
mov rax,rdx
lock xadd [rcx],al
ret
.code
ALIGN 8
PUBLIC __TBB_machine_fetchstore1
__TBB_machine_fetchstore1:
mov rax,rdx
lock xchg [rcx],al
ret
.code
ALIGN 8
PUBLIC __TBB_machine_cmpswp1
__TBB_machine_cmpswp1:
mov rax,r8
lock cmpxchg [rcx],dl
ret
.code
ALIGN 8
PUBLIC __TBB_machine_fetchadd2
__TBB_machine_fetchadd2:
mov rax,rdx
lock xadd [rcx],ax
ret
.code
ALIGN 8
PUBLIC __TBB_machine_fetchstore2
__TBB_machine_fetchstore2:
mov rax,rdx
lock xchg [rcx],ax
ret
.code
ALIGN 8
PUBLIC __TBB_machine_cmpswp2
__TBB_machine_cmpswp2:
mov rax,r8
lock cmpxchg [rcx],dx
ret
.code
ALIGN 8
PUBLIC __TBB_machine_pause
__TBB_machine_pause:
L1:
dw 090f3H; pause
add ecx,-1
jne L1
ret
end

View File

@@ -0,0 +1,33 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
.code
ALIGN 8
PUBLIC __TBB_get_cpu_ctl_env
__TBB_get_cpu_ctl_env:
stmxcsr [rcx]
fstcw [rcx+4]
ret
.code
ALIGN 8
PUBLIC __TBB_set_cpu_ctl_env
__TBB_set_cpu_ctl_env:
ldmxcsr [rcx]
fldcw [rcx+4]
ret
end

View File

@@ -0,0 +1,76 @@
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
;
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
; you can redistribute it and/or modify it under the terms of the GNU General Public License
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
; See the GNU General Public License for more details. You should have received a copy of
; the GNU General Public License along with Threading Building Blocks; if not, write to the
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
;
; As a special exception, you may use this file as part of a free software library without
; restriction. Specifically, if other files instantiate templates or use macros or inline
; functions from this file, or you compile this file and link it with other files to produce
; an executable, this file does not by itself cause the resulting executable to be covered
; by the GNU General Public License. This exception does not however invalidate any other
; reasons why the executable file might be covered by the GNU General Public License.
.code
ALIGN 8
PUBLIC __TBB_machine_try_lock_elided
__TBB_machine_try_lock_elided:
xor rax, rax
mov al, 1
BYTE 0F2H
xchg al, byte ptr [rcx]
xor al, 1
ret
.code
ALIGN 8
PUBLIC __TBB_machine_unlock_elided
__TBB_machine_unlock_elided:
BYTE 0F3H
mov byte ptr [rcx], 0
ret
.code
ALIGN 8
PUBLIC __TBB_machine_begin_transaction
__TBB_machine_begin_transaction:
mov eax, -1
BYTE 0C7H
BYTE 0F8H
BYTE 000H
BYTE 000H
BYTE 000H
BYTE 000H
ret
.code
ALIGN 8
PUBLIC __TBB_machine_end_transaction
__TBB_machine_end_transaction:
BYTE 00FH
BYTE 001H
BYTE 0D5H
ret
.code
ALIGN 8
PUBLIC __TBB_machine_transaction_conflict_abort
__TBB_machine_transaction_conflict_abort:
BYTE 0C6H
BYTE 0F8H
BYTE 0FFH ; 12.4.5 Abort argument: lock not free when tested
ret
.code
ALIGN 8
PUBLIC __TBB_machine_is_in_transaction
__TBB_machine_is_in_transaction:
xor eax, eax
BYTE 00FH ; _xtest sets or clears ZF
BYTE 001H
BYTE 0D6H
jz rset
mov al,1
rset:
ret
end

View File

@@ -0,0 +1,180 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__aggregator_impl_H
#define __TBB__aggregator_impl_H
#include "../atomic.h"
#if !__TBBMALLOC_BUILD
#include "../tbb_profiling.h"
#endif
namespace tbb {
namespace interface6 {
namespace internal {
using namespace tbb::internal;
//! aggregated_operation base class
template <typename Derived>
class aggregated_operation {
public:
uintptr_t status;
Derived *next;
aggregated_operation() : status(0), next(NULL) {}
};
//! Aggregator base class
/** An aggregator for collecting operations coming from multiple sources and executing
them serially on a single thread. operation_type must be derived from
aggregated_operation. The parameter handler_type is a functor that will be passed the
list of operations and is expected to handle each operation appropriately, setting the
status of each operation to non-zero.*/
template < typename operation_type >
class aggregator_generic {
public:
aggregator_generic() : handler_busy(false) { pending_operations = NULL; }
//! Place operation in list
/** Place operation in list and either handle list or wait for operation to
complete.
long_life_time specifies life time of an operation inserting in an aggregator.
"Long" (long_life_time == true) life time operation can be accessed
even after executing it.
"Short" (long_life_time == false) life time operations can be destroyed
during executing so any access to it after executing is invalid.*/
template < typename handler_type >
void execute(operation_type *op, handler_type &handle_operations, bool long_life_time = true) {
operation_type *res;
// op->status should be read before inserting the operation in the
// aggregator queue since it can become invalid after executing a
// handler (if the operation has 'short' life time.)
const uintptr_t status = op->status;
// ITT note: &(op->status) tag is used to cover accesses to this op node. This
// thread has created the operation, and now releases it so that the handler
// thread may handle the associated operation w/o triggering a race condition;
// thus this tag will be acquired just before the operation is handled in the
// handle_operations functor.
call_itt_notify(releasing, &(op->status));
// insert the operation in the queue.
do {
// ITT may flag the following line as a race; it is a false positive:
// This is an atomic read; we don't provide itt_hide_load_word for atomics
op->next = res = pending_operations; // NOT A RACE
} while (pending_operations.compare_and_swap(op, res) != res);
if (!res) { // first in the list; handle the operations.
// ITT note: &pending_operations tag covers access to the handler_busy flag,
// which this waiting handler thread will try to set before entering
// handle_operations.
call_itt_notify(acquired, &pending_operations);
start_handle_operations(handle_operations);
// The operation with 'short' life time can already be destroyed.
if (long_life_time)
__TBB_ASSERT(op->status, NULL);
}
// not first; wait for op to be ready.
else if (!status) { // operation is blocking here.
__TBB_ASSERT(long_life_time, "The blocking operation cannot have 'short' life time. Since it can already be destroyed.");
call_itt_notify(prepare, &(op->status));
spin_wait_while_eq(op->status, uintptr_t(0));
itt_load_word_with_acquire(op->status);
}
}
private:
//! An atomically updated list (aka mailbox) of pending operations
atomic<operation_type *> pending_operations;
//! Controls thread access to handle_operations
uintptr_t handler_busy;
//! Trigger the handling of operations when the handler is free
template < typename handler_type >
void start_handle_operations( handler_type &handle_operations ) {
operation_type *op_list;
// ITT note: &handler_busy tag covers access to pending_operations as it is passed
// between active and waiting handlers. Below, the waiting handler waits until
// the active handler releases, and the waiting handler acquires &handler_busy as
// it becomes the active_handler. The release point is at the end of this
// function, when all operations in pending_operations have been handled by the
// owner of this aggregator.
call_itt_notify(prepare, &handler_busy);
// get the handler_busy:
// only one thread can possibly spin here at a time
spin_wait_until_eq(handler_busy, uintptr_t(0));
call_itt_notify(acquired, &handler_busy);
// acquire fence not necessary here due to causality rule and surrounding atomics
__TBB_store_with_release(handler_busy, uintptr_t(1));
// ITT note: &pending_operations tag covers access to the handler_busy flag
// itself. Capturing the state of the pending_operations signifies that
// handler_busy has been set and a new active handler will now process that list's
// operations.
call_itt_notify(releasing, &pending_operations);
// grab pending_operations
op_list = pending_operations.fetch_and_store(NULL);
// handle all the operations
handle_operations(op_list);
// release the handler
itt_store_word_with_release(handler_busy, uintptr_t(0));
}
};
template < typename handler_type, typename operation_type >
class aggregator : public aggregator_generic<operation_type> {
handler_type handle_operations;
public:
aggregator() {}
explicit aggregator(handler_type h) : handle_operations(h) {}
void initialize_handler(handler_type h) { handle_operations = h; }
void execute(operation_type *op) {
aggregator_generic<operation_type>::execute(op, handle_operations);
}
};
// the most-compatible friend declaration (vs, gcc, icc) is
// template<class U, class V> friend class aggregating_functor;
template<typename aggregating_class, typename operation_list>
class aggregating_functor {
aggregating_class *fi;
public:
aggregating_functor() {}
aggregating_functor(aggregating_class *fi_) : fi(fi_) {}
void operator()(operation_list* op_list) { fi->handle_operations(op_list); }
};
} // namespace internal
} // namespace interface6
namespace internal {
using interface6::internal::aggregated_operation;
using interface6::internal::aggregator_generic;
using interface6::internal::aggregator;
using interface6::internal::aggregating_functor;
} // namespace internal
} // namespace tbb
#endif // __TBB__aggregator_impl_H

View File

@@ -0,0 +1,757 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__flow_graph_impl_H
#define __TBB__flow_graph_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
namespace internal {
namespace graph_policy_namespace {
enum graph_buffer_policy { rejecting, reserving, queueing, tag_matching };
}
// -------------- function_body containers ----------------------
//! A functor that takes no input and generates a value of type Output
template< typename Output >
class source_body : tbb::internal::no_assign {
public:
virtual ~source_body() {}
virtual bool operator()(Output &output) = 0;
virtual source_body* clone() = 0;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
virtual void reset_body() = 0;
#endif
};
//! The leaf for source_body
template< typename Output, typename Body>
class source_body_leaf : public source_body<Output> {
public:
source_body_leaf( const Body &_body ) : body(_body), init_body(_body) { }
/*override*/ bool operator()(Output &output) { return body( output ); }
/*override*/ source_body_leaf* clone() {
return new source_body_leaf< Output, Body >(init_body);
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
Body get_body() { return body; }
private:
Body body;
Body init_body;
};
//! A functor that takes an Input and generates an Output
template< typename Input, typename Output >
class function_body : tbb::internal::no_assign {
public:
virtual ~function_body() {}
virtual Output operator()(const Input &input) = 0;
virtual function_body* clone() = 0;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
virtual void reset_body() = 0;
#endif
};
//! the leaf for function_body
template <typename Input, typename Output, typename B>
class function_body_leaf : public function_body< Input, Output > {
public:
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
Output operator()(const Input &i) { return body(i); }
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
B get_body() { return body; }
/*override*/ function_body_leaf* clone() {
return new function_body_leaf< Input, Output, B >(init_body);
}
private:
B body;
B init_body;
};
//! the leaf for function_body specialized for Input and output of continue_msg
template <typename B>
class function_body_leaf< continue_msg, continue_msg, B> : public function_body< continue_msg, continue_msg > {
public:
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
continue_msg operator()( const continue_msg &i ) {
body(i);
return i;
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
B get_body() { return body; }
/*override*/ function_body_leaf* clone() {
return new function_body_leaf< continue_msg, continue_msg, B >(init_body);
}
private:
B body;
B init_body;
};
//! the leaf for function_body specialized for Output of continue_msg
template <typename Input, typename B>
class function_body_leaf< Input, continue_msg, B> : public function_body< Input, continue_msg > {
public:
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
continue_msg operator()(const Input &i) {
body(i);
return continue_msg();
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
B get_body() { return body; }
/*override*/ function_body_leaf* clone() {
return new function_body_leaf< Input, continue_msg, B >(init_body);
}
private:
B body;
B init_body;
};
//! the leaf for function_body specialized for Input of continue_msg
template <typename Output, typename B>
class function_body_leaf< continue_msg, Output, B > : public function_body< continue_msg, Output > {
public:
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
Output operator()(const continue_msg &i) {
return body(i);
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
B get_body() { return body; }
/*override*/ function_body_leaf* clone() {
return new function_body_leaf< continue_msg, Output, B >(init_body);
}
private:
B body;
B init_body;
};
//! function_body that takes an Input and a set of output ports
template<typename Input, typename OutputSet>
class multifunction_body : tbb::internal::no_assign {
public:
virtual ~multifunction_body () {}
virtual void operator()(const Input &/* input*/, OutputSet &/*oset*/) = 0;
virtual multifunction_body* clone() = 0;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
virtual void reset_body() = 0;
#endif
};
//! leaf for multifunction. OutputSet can be a std::tuple or a vector.
template<typename Input, typename OutputSet, typename B>
class multifunction_body_leaf : public multifunction_body<Input, OutputSet> {
public:
multifunction_body_leaf(const B &_body) : body(_body), init_body(_body) { }
void operator()(const Input &input, OutputSet &oset) {
body(input, oset); // body may explicitly put() to one or more of oset.
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void reset_body() {
body = init_body;
}
#endif
B get_body() { return body; }
/*override*/ multifunction_body_leaf* clone() {
return new multifunction_body_leaf<Input, OutputSet,B>(init_body);
}
private:
B body;
B init_body;
};
// --------------------------- end of function_body containers ------------------------
// --------------------------- node task bodies ---------------------------------------
//! A task that calls a node's forward_task function
template< typename NodeType >
class forward_task_bypass : public task {
NodeType &my_node;
public:
forward_task_bypass( NodeType &n ) : my_node(n) {}
task *execute() {
task * new_task = my_node.forward_task();
if (new_task == SUCCESSFULLY_ENQUEUED) new_task = NULL;
return new_task;
}
};
//! A task that calls a node's apply_body_bypass function, passing in an input of type Input
// return the task* unless it is SUCCESSFULLY_ENQUEUED, in which case return NULL
template< typename NodeType, typename Input >
class apply_body_task_bypass : public task {
NodeType &my_node;
Input my_input;
public:
apply_body_task_bypass( NodeType &n, const Input &i ) : my_node(n), my_input(i) {}
task *execute() {
task * next_task = my_node.apply_body_bypass( my_input );
if(next_task == SUCCESSFULLY_ENQUEUED) next_task = NULL;
return next_task;
}
};
//! A task that calls a node's apply_body function with no input
template< typename NodeType >
class source_task_bypass : public task {
NodeType &my_node;
public:
source_task_bypass( NodeType &n ) : my_node(n) {}
task *execute() {
task *new_task = my_node.apply_body_bypass( );
if(new_task == SUCCESSFULLY_ENQUEUED) return NULL;
return new_task;
}
};
// ------------------------ end of node task bodies -----------------------------------
//! An empty functor that takes an Input and returns a default constructed Output
template< typename Input, typename Output >
struct empty_body {
Output operator()( const Input & ) const { return Output(); }
};
//! A node_cache maintains a std::queue of elements of type T. Each operation is protected by a lock.
template< typename T, typename M=spin_mutex >
class node_cache {
public:
typedef size_t size_type;
bool empty() {
typename my_mutex_type::scoped_lock lock( my_mutex );
return internal_empty();
}
void add( T &n ) {
typename my_mutex_type::scoped_lock lock( my_mutex );
internal_push(n);
}
void remove( T &n ) {
typename my_mutex_type::scoped_lock lock( my_mutex );
for ( size_t i = internal_size(); i != 0; --i ) {
T &s = internal_pop();
if ( &s == &n ) return; // only remove one predecessor per request
internal_push(s);
}
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<T *> predecessor_vector_type;
void internal_add_built_predecessor( T &n ) {
typename my_mutex_type::scoped_lock lock( my_mutex );
my_built_predecessors.add_edge(n);
}
void internal_delete_built_predecessor( T &n ) {
typename my_mutex_type::scoped_lock lock( my_mutex );
my_built_predecessors.delete_edge(n);
}
void copy_predecessors( predecessor_vector_type &v) {
typename my_mutex_type::scoped_lock lock( my_mutex );
my_built_predecessors.copy_edges(v);
}
size_t predecessor_count() {
typename my_mutex_type::scoped_lock lock(my_mutex);
return (size_t)(my_built_predecessors.edge_count());
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
protected:
typedef M my_mutex_type;
my_mutex_type my_mutex;
std::queue< T * > my_q;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
edge_container<T> my_built_predecessors;
#endif
// Assumes lock is held
inline bool internal_empty( ) {
return my_q.empty();
}
// Assumes lock is held
inline size_type internal_size( ) {
return my_q.size();
}
// Assumes lock is held
inline void internal_push( T &n ) {
my_q.push(&n);
}
// Assumes lock is held
inline T &internal_pop() {
T *v = my_q.front();
my_q.pop();
return *v;
}
};
//! A cache of predecessors that only supports try_get
template< typename T, typename M=spin_mutex >
class predecessor_cache : public node_cache< sender<T>, M > {
public:
typedef M my_mutex_type;
typedef T output_type;
typedef sender<output_type> predecessor_type;
typedef receiver<output_type> successor_type;
predecessor_cache( ) : my_owner( NULL ) { }
void set_owner( successor_type *owner ) { my_owner = owner; }
bool get_item( output_type &v ) {
bool msg = false;
do {
predecessor_type *src;
{
typename my_mutex_type::scoped_lock lock(this->my_mutex);
if ( this->internal_empty() ) {
break;
}
src = &this->internal_pop();
}
// Try to get from this sender
msg = src->try_get( v );
if (msg == false) {
// Relinquish ownership of the edge
if ( my_owner)
src->register_successor( *my_owner );
} else {
// Retain ownership of the edge
this->add(*src);
}
} while ( msg == false );
return msg;
}
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
if(my_owner) {
for(;;) {
predecessor_type *src;
{
if(this->internal_empty()) break;
src = &this->internal_pop();
}
src->register_successor( *my_owner);
}
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
if (f&rf_extract && my_owner)
my_built_predecessors.receiver_extract(*my_owner);
__TBB_ASSERT(!(f&rf_extract) || this->internal_empty(), "predecessor cache not empty");
#endif
}
protected:
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
using node_cache< sender<T>, M >::my_built_predecessors;
#endif
successor_type *my_owner;
};
//! An cache of predecessors that supports requests and reservations
template< typename T, typename M=spin_mutex >
class reservable_predecessor_cache : public predecessor_cache< T, M > {
public:
typedef M my_mutex_type;
typedef T output_type;
typedef sender<T> predecessor_type;
typedef receiver<T> successor_type;
reservable_predecessor_cache( ) : reserved_src(NULL) { }
bool
try_reserve( output_type &v ) {
bool msg = false;
do {
{
typename my_mutex_type::scoped_lock lock(this->my_mutex);
if ( reserved_src || this->internal_empty() )
return false;
reserved_src = &this->internal_pop();
}
// Try to get from this sender
msg = reserved_src->try_reserve( v );
if (msg == false) {
typename my_mutex_type::scoped_lock lock(this->my_mutex);
// Relinquish ownership of the edge
reserved_src->register_successor( *this->my_owner );
reserved_src = NULL;
} else {
// Retain ownership of the edge
this->add( *reserved_src );
}
} while ( msg == false );
return msg;
}
bool
try_release( ) {
reserved_src->try_release( );
reserved_src = NULL;
return true;
}
bool
try_consume( ) {
reserved_src->try_consume( );
reserved_src = NULL;
return true;
}
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
reserved_src = NULL;
predecessor_cache<T,M>::reset(__TBB_PFG_RESET_ARG(f));
}
private:
predecessor_type *reserved_src;
};
//! An abstract cache of successors
template<typename T, typename M=spin_rw_mutex >
class successor_cache : tbb::internal::no_copy {
protected:
typedef M my_mutex_type;
my_mutex_type my_mutex;
typedef receiver<T> *pointer_type;
typedef std::list< pointer_type > my_successors_type;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
edge_container<receiver<T> > my_built_successors;
#endif
my_successors_type my_successors;
sender<T> *my_owner;
public:
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<pointer_type> successor_vector_type;
void internal_add_built_successor( receiver<T> &r) {
typename my_mutex_type::scoped_lock l(my_mutex, true);
my_built_successors.add_edge( r );
}
void internal_delete_built_successor( receiver<T> &r) {
typename my_mutex_type::scoped_lock l(my_mutex, true);
my_built_successors.delete_edge(r);
}
void copy_successors( successor_vector_type &v) {
typename my_mutex_type::scoped_lock l(my_mutex, false);
my_built_successors.copy_edges(v);
}
size_t successor_count() {
typename my_mutex_type::scoped_lock l(my_mutex,false);
return my_built_successors.edge_count();
}
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
if (f&rf_extract && my_owner)
my_built_successors.sender_extract(*my_owner);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
successor_cache( ) : my_owner(NULL) {}
void set_owner( sender<T> *owner ) { my_owner = owner; }
virtual ~successor_cache() {}
void register_successor( receiver<T> &r ) {
typename my_mutex_type::scoped_lock l(my_mutex, true);
my_successors.push_back( &r );
}
void remove_successor( receiver<T> &r ) {
typename my_mutex_type::scoped_lock l(my_mutex, true);
for ( typename my_successors_type::iterator i = my_successors.begin();
i != my_successors.end(); ++i ) {
if ( *i == & r ) {
my_successors.erase(i);
break;
}
}
}
bool empty() {
typename my_mutex_type::scoped_lock l(my_mutex, false);
return my_successors.empty();
}
void clear() {
my_successors.clear();
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
my_built_successors.clear();
#endif
}
virtual task * try_put_task( const T &t ) = 0;
};
//! An abstract cache of successors, specialized to continue_msg
template<>
class successor_cache< continue_msg > : tbb::internal::no_copy {
protected:
typedef spin_rw_mutex my_mutex_type;
my_mutex_type my_mutex;
typedef receiver<continue_msg> *pointer_type;
typedef std::list< pointer_type > my_successors_type;
my_successors_type my_successors;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
edge_container<receiver<continue_msg> > my_built_successors;
#endif
sender<continue_msg> *my_owner;
public:
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<pointer_type> successor_vector_type;
void internal_add_built_successor( receiver<continue_msg> &r) {
my_mutex_type::scoped_lock l(my_mutex, true);
my_built_successors.add_edge( r );
}
void internal_delete_built_successor( receiver<continue_msg> &r) {
my_mutex_type::scoped_lock l(my_mutex, true);
my_built_successors.delete_edge(r);
}
void copy_successors( successor_vector_type &v) {
my_mutex_type::scoped_lock l(my_mutex, false);
my_built_successors.copy_edges(v);
}
size_t successor_count() {
my_mutex_type::scoped_lock l(my_mutex,false);
return my_built_successors.edge_count();
}
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
if (f&rf_extract && my_owner)
my_built_successors.sender_extract(*my_owner);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
successor_cache( ) : my_owner(NULL) {}
void set_owner( sender<continue_msg> *owner ) { my_owner = owner; }
virtual ~successor_cache() {}
void register_successor( receiver<continue_msg> &r ) {
my_mutex_type::scoped_lock l(my_mutex, true);
my_successors.push_back( &r );
if ( my_owner && r.is_continue_receiver() ) {
r.register_predecessor( *my_owner );
}
}
void remove_successor( receiver<continue_msg> &r ) {
my_mutex_type::scoped_lock l(my_mutex, true);
for ( my_successors_type::iterator i = my_successors.begin();
i != my_successors.end(); ++i ) {
if ( *i == & r ) {
// TODO: Check if we need to test for continue_receiver before
// removing from r.
if ( my_owner )
r.remove_predecessor( *my_owner );
my_successors.erase(i);
break;
}
}
}
bool empty() {
my_mutex_type::scoped_lock l(my_mutex, false);
return my_successors.empty();
}
void clear() {
my_successors.clear();
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
my_built_successors.clear();
#endif
}
virtual task * try_put_task( const continue_msg &t ) = 0;
};
//! A cache of successors that are broadcast to
template<typename T, typename M=spin_rw_mutex>
class broadcast_cache : public successor_cache<T, M> {
typedef M my_mutex_type;
typedef std::list< receiver<T> * > my_successors_type;
public:
broadcast_cache( ) {}
// as above, but call try_put_task instead, and return the last task we received (if any)
/*override*/ task * try_put_task( const T &t ) {
task * last_task = NULL;
bool upgraded = true;
typename my_mutex_type::scoped_lock l(this->my_mutex, upgraded);
typename my_successors_type::iterator i = this->my_successors.begin();
while ( i != this->my_successors.end() ) {
task *new_task = (*i)->try_put_task(t);
last_task = combine_tasks(last_task, new_task); // enqueue if necessary
if(new_task) {
++i;
}
else { // failed
if ( (*i)->register_predecessor(*this->my_owner) ) {
if (!upgraded) {
l.upgrade_to_writer();
upgraded = true;
}
i = this->my_successors.erase(i);
} else {
++i;
}
}
}
return last_task;
}
};
//! A cache of successors that are put in a round-robin fashion
template<typename T, typename M=spin_rw_mutex >
class round_robin_cache : public successor_cache<T, M> {
typedef size_t size_type;
typedef M my_mutex_type;
typedef std::list< receiver<T> * > my_successors_type;
public:
round_robin_cache( ) {}
size_type size() {
typename my_mutex_type::scoped_lock l(this->my_mutex, false);
return this->my_successors.size();
}
/*override*/task *try_put_task( const T &t ) {
bool upgraded = true;
typename my_mutex_type::scoped_lock l(this->my_mutex, upgraded);
typename my_successors_type::iterator i = this->my_successors.begin();
while ( i != this->my_successors.end() ) {
task *new_task = (*i)->try_put_task(t);
if ( new_task ) {
return new_task;
} else {
if ( (*i)->register_predecessor(*this->my_owner) ) {
if (!upgraded) {
l.upgrade_to_writer();
upgraded = true;
}
i = this->my_successors.erase(i);
}
else {
++i;
}
}
}
return NULL;
}
};
template<typename T>
class decrementer : public continue_receiver, tbb::internal::no_copy {
T *my_node;
task *execute() {
return my_node->decrement_counter();
}
public:
typedef continue_msg input_type;
typedef continue_msg output_type;
decrementer( int number_of_predecessors = 0 ) : continue_receiver( number_of_predecessors ) { }
void set_owner( T *node ) { my_node = node; }
};
}
#endif // __TBB__flow_graph_impl_H

View File

@@ -0,0 +1,453 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__flow_graph_indexer_impl_H
#define __TBB__flow_graph_indexer_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#include "tbb/internal/_flow_graph_types_impl.h"
namespace internal {
// Output of the indexer_node is a tbb::flow::tagged_msg, and will be of
// the form tagged_msg<tag, result>
// where the value of tag will indicate which result was put to the
// successor.
template<typename IndexerNodeBaseType, typename T, size_t K>
task* do_try_put(const T &v, void *p) {
typename IndexerNodeBaseType::output_type o(K, v);
return reinterpret_cast<IndexerNodeBaseType *>(p)->try_put_task(&o);
}
template<typename TupleTypes,int N>
struct indexer_helper {
template<typename IndexerNodeBaseType, typename PortTuple>
static inline void set_indexer_node_pointer(PortTuple &my_input, IndexerNodeBaseType *p) {
typedef typename tuple_element<N-1, TupleTypes>::type T;
task *(*indexer_node_put_task)(const T&, void *) = do_try_put<IndexerNodeBaseType, T, N-1>;
tbb::flow::get<N-1>(my_input).set_up(p, indexer_node_put_task);
indexer_helper<TupleTypes,N-1>::template set_indexer_node_pointer<IndexerNodeBaseType,PortTuple>(my_input, p);
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
template<typename InputTuple>
static inline void reset_inputs(InputTuple &my_input, reset_flags f) {
join_helper<N-1>::reset_inputs(my_input, f);
tbb::flow::get<N-1>(my_input).reset_receiver(f);
}
#endif
};
template<typename TupleTypes>
struct indexer_helper<TupleTypes,1> {
template<typename IndexerNodeBaseType, typename PortTuple>
static inline void set_indexer_node_pointer(PortTuple &my_input, IndexerNodeBaseType *p) {
typedef typename tuple_element<0, TupleTypes>::type T;
task *(*indexer_node_put_task)(const T&, void *) = do_try_put<IndexerNodeBaseType, T, 0>;
tbb::flow::get<0>(my_input).set_up(p, indexer_node_put_task);
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
template<typename InputTuple>
static inline void reset_inputs(InputTuple &my_input, reset_flags f) {
tbb::flow::get<0>(my_input).reset_receiver(f);
}
#endif
};
template<typename T>
class indexer_input_port : public receiver<T> {
private:
void* my_indexer_ptr;
typedef task* (* forward_function_ptr)(T const &, void* );
forward_function_ptr my_try_put_task;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
spin_mutex my_pred_mutex;
edge_container<sender<T> > my_built_predecessors;
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
public:
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
indexer_input_port() : my_pred_mutex() {}
indexer_input_port( const indexer_input_port & /*other*/ ) : receiver<T>(), my_pred_mutex() {
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
void set_up(void *p, forward_function_ptr f) {
my_indexer_ptr = p;
my_try_put_task = f;
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<sender<T> *> predecessor_vector_type;
/*override*/size_t predecessor_count() {
spin_mutex::scoped_lock l(my_pred_mutex);
return my_built_predecessors.edge_count();
}
/*override*/void internal_add_built_predecessor(sender<T> &p) {
spin_mutex::scoped_lock l(my_pred_mutex);
my_built_predecessors.add_edge(p);
}
/*override*/void internal_delete_built_predecessor(sender<T> &p) {
spin_mutex::scoped_lock l(my_pred_mutex);
my_built_predecessors.delete_edge(p);
}
/*override*/void copy_predecessors( predecessor_vector_type &v) {
spin_mutex::scoped_lock l(my_pred_mutex);
return my_built_predecessors.copy_edges(v);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
protected:
template< typename R, typename B > friend class run_and_put_task;
template<typename X, typename Y> friend class internal::broadcast_cache;
template<typename X, typename Y> friend class internal::round_robin_cache;
task *try_put_task(const T &v) {
return my_try_put_task(v, my_indexer_ptr);
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
public:
/*override*/void reset_receiver(__TBB_PFG_RESET_ARG(reset_flags f)) {
if(f&rf_extract) my_built_predecessors.receiver_extract(*this);
}
#else
/*override*/void reset_receiver(__TBB_PFG_RESET_ARG(reset_flags /*f*/)) { }
#endif
};
template<typename InputTuple, typename OutputType, typename StructTypes>
class indexer_node_FE {
public:
static const int N = tbb::flow::tuple_size<InputTuple>::value;
typedef OutputType output_type;
typedef InputTuple input_type;
input_type &input_ports() { return my_inputs; }
protected:
input_type my_inputs;
};
//! indexer_node_base
template<typename InputTuple, typename OutputType, typename StructTypes>
class indexer_node_base : public graph_node, public indexer_node_FE<InputTuple, OutputType,StructTypes>,
public sender<OutputType> {
protected:
using graph_node::my_graph;
public:
static const size_t N = tbb::flow::tuple_size<InputTuple>::value;
typedef OutputType output_type;
typedef StructTypes tuple_types;
typedef receiver<output_type> successor_type;
typedef indexer_node_FE<InputTuple, output_type,StructTypes> input_ports_type;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<successor_type *> successor_vector_type;
#endif
private:
// ----------- Aggregator ------------
enum op_type { reg_succ, rem_succ, try__put_task
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
, add_blt_succ, del_blt_succ,
blt_succ_cnt, blt_succ_cpy
#endif
};
enum op_stat {WAIT=0, SUCCEEDED, FAILED};
typedef indexer_node_base<InputTuple,output_type,StructTypes> my_class;
class indexer_node_base_operation : public aggregated_operation<indexer_node_base_operation> {
public:
char type;
union {
output_type const *my_arg;
successor_type *my_succ;
task *bypass_t;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
size_t cnt_val;
successor_vector_type *succv;
#endif
};
indexer_node_base_operation(const output_type* e, op_type t) :
type(char(t)), my_arg(e) {}
indexer_node_base_operation(const successor_type &s, op_type t) : type(char(t)),
my_succ(const_cast<successor_type *>(&s)) {}
indexer_node_base_operation(op_type t) : type(char(t)) {}
};
typedef internal::aggregating_functor<my_class, indexer_node_base_operation> my_handler;
friend class internal::aggregating_functor<my_class, indexer_node_base_operation>;
aggregator<my_handler, indexer_node_base_operation> my_aggregator;
void handle_operations(indexer_node_base_operation* op_list) {
indexer_node_base_operation *current;
while(op_list) {
current = op_list;
op_list = op_list->next;
switch(current->type) {
case reg_succ:
my_successors.register_successor(*(current->my_succ));
__TBB_store_with_release(current->status, SUCCEEDED);
break;
case rem_succ:
my_successors.remove_successor(*(current->my_succ));
__TBB_store_with_release(current->status, SUCCEEDED);
break;
case try__put_task: {
current->bypass_t = my_successors.try_put_task(*(current->my_arg));
__TBB_store_with_release(current->status, SUCCEEDED); // return of try_put_task actual return value
}
break;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
case add_blt_succ:
my_successors.internal_add_built_successor(*(current->my_succ));
__TBB_store_with_release(current->status, SUCCEEDED);
break;
case del_blt_succ:
my_successors.internal_delete_built_successor(*(current->my_succ));
__TBB_store_with_release(current->status, SUCCEEDED);
break;
case blt_succ_cnt:
current->cnt_val = my_successors.successor_count();
__TBB_store_with_release(current->status, SUCCEEDED);
break;
case blt_succ_cpy:
my_successors.copy_successors(*(current->succv));
__TBB_store_with_release(current->status, SUCCEEDED);
break;
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
}
}
}
// ---------- end aggregator -----------
public:
indexer_node_base(graph& g) : graph_node(g), input_ports_type() {
indexer_helper<StructTypes,N>::set_indexer_node_pointer(this->my_inputs, this);
my_successors.set_owner(this);
my_aggregator.initialize_handler(my_handler(this));
}
indexer_node_base(const indexer_node_base& other) : graph_node(other.my_graph), input_ports_type(), sender<output_type>() {
indexer_helper<StructTypes,N>::set_indexer_node_pointer(this->my_inputs, this);
my_successors.set_owner(this);
my_aggregator.initialize_handler(my_handler(this));
}
bool register_successor(successor_type &r) {
indexer_node_base_operation op_data(r, reg_succ);
my_aggregator.execute(&op_data);
return op_data.status == SUCCEEDED;
}
bool remove_successor( successor_type &r) {
indexer_node_base_operation op_data(r, rem_succ);
my_aggregator.execute(&op_data);
return op_data.status == SUCCEEDED;
}
task * try_put_task(output_type const *v) {
indexer_node_base_operation op_data(v, try__put_task);
my_aggregator.execute(&op_data);
return op_data.bypass_t;
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
void internal_add_built_successor( successor_type &r) {
indexer_node_base_operation op_data(r, add_blt_succ);
my_aggregator.execute(&op_data);
}
void internal_delete_built_successor( successor_type &r) {
indexer_node_base_operation op_data(r, del_blt_succ);
my_aggregator.execute(&op_data);
}
size_t successor_count() {
indexer_node_base_operation op_data(blt_succ_cnt);
my_aggregator.execute(&op_data);
return op_data.cnt_val;
}
void copy_successors( successor_vector_type &v) {
indexer_node_base_operation op_data(blt_succ_cpy);
op_data.succv = &v;
my_aggregator.execute(&op_data);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
protected:
/*override*/void reset(__TBB_PFG_RESET_ARG(reset_flags f)) {
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
my_successors.reset(f);
indexer_helper<StructTypes,N>::reset_inputs(this->my_inputs, f);
#endif
}
private:
broadcast_cache<output_type, null_rw_mutex> my_successors;
}; //indexer_node_base
template<int N, typename InputTuple> struct input_types;
template<typename InputTuple>
struct input_types<1, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename internal::tagged_msg<size_t, first_type > type;
};
template<typename InputTuple>
struct input_types<2, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type> type;
};
template<typename InputTuple>
struct input_types<3, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type> type;
};
template<typename InputTuple>
struct input_types<4, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type> type;
};
template<typename InputTuple>
struct input_types<5, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type> type;
};
template<typename InputTuple>
struct input_types<6, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename tuple_element<5, InputTuple>::type sixth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type, sixth_type> type;
};
template<typename InputTuple>
struct input_types<7, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename tuple_element<5, InputTuple>::type sixth_type;
typedef typename tuple_element<6, InputTuple>::type seventh_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type, sixth_type,
seventh_type> type;
};
template<typename InputTuple>
struct input_types<8, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename tuple_element<5, InputTuple>::type sixth_type;
typedef typename tuple_element<6, InputTuple>::type seventh_type;
typedef typename tuple_element<7, InputTuple>::type eighth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type, sixth_type,
seventh_type, eighth_type> type;
};
template<typename InputTuple>
struct input_types<9, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename tuple_element<5, InputTuple>::type sixth_type;
typedef typename tuple_element<6, InputTuple>::type seventh_type;
typedef typename tuple_element<7, InputTuple>::type eighth_type;
typedef typename tuple_element<8, InputTuple>::type nineth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type, sixth_type,
seventh_type, eighth_type, nineth_type> type;
};
template<typename InputTuple>
struct input_types<10, InputTuple> {
typedef typename tuple_element<0, InputTuple>::type first_type;
typedef typename tuple_element<1, InputTuple>::type second_type;
typedef typename tuple_element<2, InputTuple>::type third_type;
typedef typename tuple_element<3, InputTuple>::type fourth_type;
typedef typename tuple_element<4, InputTuple>::type fifth_type;
typedef typename tuple_element<5, InputTuple>::type sixth_type;
typedef typename tuple_element<6, InputTuple>::type seventh_type;
typedef typename tuple_element<7, InputTuple>::type eighth_type;
typedef typename tuple_element<8, InputTuple>::type nineth_type;
typedef typename tuple_element<9, InputTuple>::type tenth_type;
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
fourth_type, fifth_type, sixth_type,
seventh_type, eighth_type, nineth_type,
tenth_type> type;
};
// type generators
template<typename OutputTuple>
struct indexer_types : public input_types<tuple_size<OutputTuple>::value, OutputTuple> {
static const int N = tbb::flow::tuple_size<OutputTuple>::value;
typedef typename input_types<N, OutputTuple>::type output_type;
typedef typename wrap_tuple_elements<N,indexer_input_port,OutputTuple>::type input_ports_type;
typedef internal::indexer_node_FE<input_ports_type,output_type,OutputTuple> indexer_FE_type;
typedef internal::indexer_node_base<input_ports_type, output_type, OutputTuple> indexer_base_type;
};
template<class OutputTuple>
class unfolded_indexer_node : public indexer_types<OutputTuple>::indexer_base_type {
public:
typedef typename indexer_types<OutputTuple>::input_ports_type input_ports_type;
typedef OutputTuple tuple_types;
typedef typename indexer_types<OutputTuple>::output_type output_type;
private:
typedef typename indexer_types<OutputTuple>::indexer_base_type base_type;
public:
unfolded_indexer_node(graph& g) : base_type(g) {}
unfolded_indexer_node(const unfolded_indexer_node &other) : base_type(other) {}
};
} /* namespace internal */
#endif /* __TBB__flow_graph_indexer_impl_H */

View File

@@ -0,0 +1,279 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__flow_graph_item_buffer_impl_H
#define __TBB__flow_graph_item_buffer_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#include "tbb/internal/_flow_graph_types_impl.h" // for aligned_pair
// in namespace tbb::flow::interface7 (included in _flow_graph_node_impl.h)
//! Expandable buffer of items. The possible operations are push, pop,
//* tests for empty and so forth. No mutual exclusion is built in.
//* objects are constructed into and explicitly-destroyed. get_my_item gives
// a read-only reference to the item in the buffer. set_my_item may be called
// with either an empty or occupied slot.
using internal::aligned_pair;
using internal::alignment_of;
namespace internal {
template <typename T, typename A=cache_aligned_allocator<T> >
class item_buffer {
public:
typedef T item_type;
enum buffer_item_state { no_item=0, has_item=1, reserved_item=2 };
protected:
typedef size_t size_type;
typedef typename aligned_pair<item_type, buffer_item_state>::type buffer_item_type;
typedef typename A::template rebind<buffer_item_type>::other allocator_type;
buffer_item_type *my_array;
size_type my_array_size;
static const size_type initial_buffer_size = 4;
size_type my_head;
size_type my_tail;
bool buffer_empty() { return my_head == my_tail; }
buffer_item_type &item(size_type i) {
__TBB_ASSERT(!(size_type(&(my_array[i&(my_array_size-1)].second))%alignment_of<buffer_item_state>::value),NULL);
__TBB_ASSERT(!(size_type(&(my_array[i&(my_array_size-1)].first))%alignment_of<item_type>::value), NULL);
return my_array[i & (my_array_size - 1) ];
}
bool my_item_valid(size_type i) { return item(i).second != no_item; }
bool my_item_reserved(size_type i) { return item(i).second == reserved_item; }
// object management in buffer
const item_type &get_my_item(size_t i) {
__TBB_ASSERT(my_item_valid(i),"attempt to get invalid item");
item_type *itm = (tbb::internal::punned_cast<item_type *>(&(item(i).first)));
return *(const item_type *)itm;
}
// may be called with an empty slot or a slot that has already been constructed into.
void set_my_item(size_t i, const item_type &o) {
if(item(i).second != no_item) {
destroy_item(i);
}
new(&(item(i).first)) item_type(o);
item(i).second = has_item;
}
// destructively-fetch an object from the buffer
void fetch_item(size_t i, item_type &o) {
__TBB_ASSERT(my_item_valid(i), "Trying to fetch an empty slot");
o = get_my_item(i); // could have std::move assign semantics
destroy_item(i);
}
// move an existing item from one slot to another. The moved-to slot must be unoccupied,
// the moved-from slot must exist and not be reserved. The after, from will be empty,
// to will be occupied but not reserved
void move_item(size_t to, size_t from) {
__TBB_ASSERT(!my_item_valid(to), "Trying to move to a non-empty slot");
__TBB_ASSERT(my_item_valid(from), "Trying to move from an empty slot");
set_my_item(to, get_my_item(from)); // could have std::move semantics
destroy_item(from);
}
// put an item in an empty slot. Return true if successful, else false
bool place_item(size_t here, const item_type &me) {
#if !TBB_DEPRECATED_SEQUENCER_DUPLICATES
if(my_item_valid(here)) return false;
#endif
set_my_item(here, me);
return true;
}
// could be implemented with std::move semantics
void swap_items(size_t i, size_t j) {
__TBB_ASSERT(my_item_valid(i) && my_item_valid(j), "attempt to swap invalid item(s)");
item_type temp = get_my_item(i);
set_my_item(i, get_my_item(j));
set_my_item(j, temp);
}
void destroy_item(size_type i) {
__TBB_ASSERT(my_item_valid(i), "destruction of invalid item");
(tbb::internal::punned_cast<item_type *>(&(item(i).first)))->~item_type();
item(i).second = no_item;
}
// returns a copy of the front
void copy_front(item_type &v) {
__TBB_ASSERT(my_item_valid(my_head), "attempt to fetch head non-item");
v = get_my_item(my_head);
}
// returns a copy of the back
void copy_back(item_type &v) {
__TBB_ASSERT(my_item_valid(my_tail-1), "attempt to fetch head non-item");
v = get_my_item(my_tail-1);
}
// following methods are for reservation of the front of a bufffer.
void reserve_item(size_type i) { __TBB_ASSERT(my_item_valid(i) && !my_item_reserved(i), "item cannot be reserved"); item(i).second = reserved_item; }
void release_item(size_type i) { __TBB_ASSERT(my_item_reserved(i), "item is not reserved"); item(i).second = has_item; }
void destroy_front() { destroy_item(my_head); ++my_head; }
void destroy_back() { destroy_item(my_tail-1); --my_tail; }
// we have to be able to test against a new tail value without changing my_tail
// grow_array doesn't work if we change my_tail when the old array is too small
size_type size(size_t new_tail = 0) { return (new_tail ? new_tail : my_tail) - my_head; }
size_type capacity() { return my_array_size; }
// sequencer_node does not use this method, so we don't
// need a version that passes in the new_tail value.
bool buffer_full() { return size() >= capacity(); }
//! Grows the internal array.
void grow_my_array( size_t minimum_size ) {
// test that we haven't made the structure inconsistent.
__TBB_ASSERT(capacity() >= my_tail - my_head, "total items exceed capacity");
size_type new_size = my_array_size ? 2*my_array_size : initial_buffer_size;
while( new_size<minimum_size )
new_size*=2;
buffer_item_type* new_array = allocator_type().allocate(new_size);
// initialize validity to "no"
for( size_type i=0; i<new_size; ++i ) { new_array[i].second = no_item; }
for( size_type i=my_head; i<my_tail; ++i) {
if(my_item_valid(i)) { // sequencer_node may have empty slots
// placement-new copy-construct; could be std::move
char *new_space = (char *)&(new_array[i&(new_size-1)].first);
(void)new(new_space) item_type(get_my_item(i));
new_array[i&(new_size-1)].second = item(i).second;
}
}
clean_up_buffer(/*reset_pointers*/false);
my_array = new_array;
my_array_size = new_size;
}
bool push_back(item_type &v) {
if(buffer_full()) {
grow_my_array(size() + 1);
}
set_my_item(my_tail, v);
++my_tail;
return true;
}
bool pop_back(item_type &v) {
if (!my_item_valid(my_tail-1)) {
return false;
}
copy_back(v);
destroy_back();
return true;
}
bool pop_front(item_type &v) {
if(!my_item_valid(my_head)) {
return false;
}
copy_front(v);
destroy_front();
return true;
}
// This is used both for reset and for grow_my_array. In the case of grow_my_array
// we want to retain the values of the head and tail.
void clean_up_buffer(bool reset_pointers) {
if (my_array) {
for( size_type i=0; i<my_array_size; ++i ) {
if(my_item_valid(i))
destroy_item(i);
}
allocator_type().deallocate(my_array,my_array_size);
}
my_array = NULL;
if(reset_pointers) {
my_head = my_tail = my_array_size = 0;
}
}
public:
//! Constructor
item_buffer( ) : my_array(NULL), my_array_size(0),
my_head(0), my_tail(0) {
grow_my_array(initial_buffer_size);
}
~item_buffer() {
clean_up_buffer(/*reset_pointers*/true);
}
void reset() { clean_up_buffer(/*reset_pointers*/true); grow_my_array(initial_buffer_size); }
};
//! item_buffer with reservable front-end. NOTE: if reserving, do not
//* complete operation with pop_front(); use consume_front().
//* No synchronization built-in.
template<typename T, typename A=cache_aligned_allocator<T> >
class reservable_item_buffer : public item_buffer<T, A> {
protected:
using item_buffer<T, A>::my_item_valid;
using item_buffer<T, A>::my_head;
public:
reservable_item_buffer() : item_buffer<T, A>(), my_reserved(false) {}
void reset() {my_reserved = false; item_buffer<T,A>::reset(); }
protected:
bool reserve_front(T &v) {
if(my_reserved || !my_item_valid(my_head)) return false;
my_reserved = true;
// reserving the head
this->copy_front(v);
this->reserve_item(this->my_head);
return true;
}
void consume_front() {
__TBB_ASSERT(my_reserved, "Attempt to consume a non-reserved item");
this->destroy_front();
my_reserved = false;
}
void release_front() {
__TBB_ASSERT(my_reserved, "Attempt to release a non-reserved item");
this->release_item(this->my_head);
my_reserved = false;
}
bool my_reserved;
};
} // namespace internal
#endif // __TBB__flow_graph_item_buffer_impl_H

View File

@@ -0,0 +1,742 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__flow_graph_node_impl_H
#define __TBB__flow_graph_node_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#include "_flow_graph_item_buffer_impl.h"
//! @cond INTERNAL
namespace internal {
using tbb::internal::aggregated_operation;
using tbb::internal::aggregating_functor;
using tbb::internal::aggregator;
template< typename T, typename A >
class function_input_queue : public item_buffer<T,A> {
public:
bool pop( T& t ) {
return this->pop_front( t );
}
bool push( T& t ) {
return this->push_back( t );
}
};
//! Input and scheduling for a function node that takes a type Input as input
// The only up-ref is apply_body_impl, which should implement the function
// call and any handling of the result.
template< typename Input, typename A, typename ImplType >
class function_input_base : public receiver<Input>, tbb::internal::no_assign {
enum op_stat {WAIT=0, SUCCEEDED, FAILED};
enum op_type {reg_pred, rem_pred, app_body, try_fwd, tryput_bypass, app_body_bypass
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
, add_blt_pred, del_blt_pred,
blt_pred_cnt, blt_pred_cpy // create vector copies of preds and succs
#endif
};
typedef function_input_base<Input, A, ImplType> my_class;
public:
//! The input type of this receiver
typedef Input input_type;
typedef sender<Input> predecessor_type;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<predecessor_type *> predecessor_vector_type;
#endif
//! Constructor for function_input_base
function_input_base( graph &g, size_t max_concurrency, function_input_queue<input_type,A> *q = NULL )
: my_graph(g), my_max_concurrency(max_concurrency), my_concurrency(0),
my_queue(q), forwarder_busy(false) {
my_predecessors.set_owner(this);
my_aggregator.initialize_handler(my_handler(this));
}
//! Copy constructor
function_input_base( const function_input_base& src, function_input_queue<input_type,A> *q = NULL ) :
receiver<Input>(), tbb::internal::no_assign(),
my_graph(src.my_graph), my_max_concurrency(src.my_max_concurrency),
my_concurrency(0), my_queue(q), forwarder_busy(false)
{
my_predecessors.set_owner(this);
my_aggregator.initialize_handler(my_handler(this));
}
//! Destructor
virtual ~function_input_base() {
if ( my_queue ) delete my_queue;
}
//! Put to the node, returning a task if available
virtual task * try_put_task( const input_type &t ) {
if ( my_max_concurrency == 0 ) {
return create_body_task( t );
} else {
my_operation op_data(t, tryput_bypass);
my_aggregator.execute(&op_data);
if(op_data.status == SUCCEEDED ) {
return op_data.bypass_t;
}
return NULL;
}
}
//! Adds src to the list of cached predecessors.
/* override */ bool register_predecessor( predecessor_type &src ) {
my_operation op_data(reg_pred);
op_data.r = &src;
my_aggregator.execute(&op_data);
return true;
}
//! Removes src from the list of cached predecessors.
/* override */ bool remove_predecessor( predecessor_type &src ) {
my_operation op_data(rem_pred);
op_data.r = &src;
my_aggregator.execute(&op_data);
return true;
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
//! Adds to list of predecessors added by make_edge
/*override*/ void internal_add_built_predecessor( predecessor_type &src) {
my_operation op_data(add_blt_pred);
op_data.r = &src;
my_aggregator.execute(&op_data);
}
//! removes from to list of predecessors (used by remove_edge)
/*override*/ void internal_delete_built_predecessor( predecessor_type &src) {
my_operation op_data(del_blt_pred);
op_data.r = &src;
my_aggregator.execute(&op_data);
}
/*override*/ size_t predecessor_count() {
my_operation op_data(blt_pred_cnt);
my_aggregator.execute(&op_data);
return op_data.cnt_val;
}
/*override*/ void copy_predecessors(predecessor_vector_type &v) {
my_operation op_data(blt_pred_cpy);
op_data.predv = &v;
my_aggregator.execute(&op_data);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
protected:
void reset_function_input_base( __TBB_PFG_RESET_ARG(reset_flags f)) {
my_concurrency = 0;
if(my_queue) {
my_queue->reset();
}
reset_receiver(__TBB_PFG_RESET_ARG(f));
forwarder_busy = false;
}
graph& my_graph;
const size_t my_max_concurrency;
size_t my_concurrency;
function_input_queue<input_type, A> *my_queue;
predecessor_cache<input_type, null_mutex > my_predecessors;
/*override*/void reset_receiver( __TBB_PFG_RESET_ARG(reset_flags f)) {
my_predecessors.reset(__TBB_PFG_RESET_ARG(f));
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
__TBB_ASSERT(!(f & rf_extract) || my_predecessors.empty(), "function_input_base reset failed");
#endif
}
private:
friend class apply_body_task_bypass< my_class, input_type >;
friend class forward_task_bypass< my_class >;
class my_operation : public aggregated_operation< my_operation > {
public:
char type;
union {
input_type *elem;
predecessor_type *r;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
size_t cnt_val;
predecessor_vector_type *predv;
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
};
tbb::task *bypass_t;
my_operation(const input_type& e, op_type t) :
type(char(t)), elem(const_cast<input_type*>(&e)) {}
my_operation(op_type t) : type(char(t)), r(NULL) {}
};
bool forwarder_busy;
typedef internal::aggregating_functor<my_class, my_operation> my_handler;
friend class internal::aggregating_functor<my_class, my_operation>;
aggregator< my_handler, my_operation > my_aggregator;
void handle_operations(my_operation *op_list) {
my_operation *tmp;
while (op_list) {
tmp = op_list;
op_list = op_list->next;
switch (tmp->type) {
case reg_pred:
my_predecessors.add(*(tmp->r));
__TBB_store_with_release(tmp->status, SUCCEEDED);
if (!forwarder_busy) {
forwarder_busy = true;
spawn_forward_task();
}
break;
case rem_pred:
my_predecessors.remove(*(tmp->r));
__TBB_store_with_release(tmp->status, SUCCEEDED);
break;
case app_body:
__TBB_ASSERT(my_max_concurrency != 0, NULL);
--my_concurrency;
__TBB_store_with_release(tmp->status, SUCCEEDED);
if (my_concurrency<my_max_concurrency) {
input_type i;
bool item_was_retrieved = false;
if ( my_queue )
item_was_retrieved = my_queue->pop(i);
else
item_was_retrieved = my_predecessors.get_item(i);
if (item_was_retrieved) {
++my_concurrency;
spawn_body_task(i);
}
}
break;
case app_body_bypass: {
task * new_task = NULL;
__TBB_ASSERT(my_max_concurrency != 0, NULL);
--my_concurrency;
if (my_concurrency<my_max_concurrency) {
input_type i;
bool item_was_retrieved = false;
if ( my_queue )
item_was_retrieved = my_queue->pop(i);
else
item_was_retrieved = my_predecessors.get_item(i);
if (item_was_retrieved) {
++my_concurrency;
new_task = create_body_task(i);
}
}
tmp->bypass_t = new_task;
__TBB_store_with_release(tmp->status, SUCCEEDED);
}
break;
case tryput_bypass: internal_try_put_task(tmp); break;
case try_fwd: internal_forward(tmp); break;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
case add_blt_pred: {
my_predecessors.internal_add_built_predecessor(*(tmp->r));
__TBB_store_with_release(tmp->status, SUCCEEDED);
}
break;
case del_blt_pred:
my_predecessors.internal_delete_built_predecessor(*(tmp->r));
__TBB_store_with_release(tmp->status, SUCCEEDED);
break;
case blt_pred_cnt:
tmp->cnt_val = my_predecessors.predecessor_count();
__TBB_store_with_release(tmp->status, SUCCEEDED);
break;
case blt_pred_cpy:
my_predecessors.copy_predecessors( *(tmp->predv) );
__TBB_store_with_release(tmp->status, SUCCEEDED);
break;
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
}
}
}
//! Put to the node, but return the task instead of enqueueing it
void internal_try_put_task(my_operation *op) {
__TBB_ASSERT(my_max_concurrency != 0, NULL);
if (my_concurrency < my_max_concurrency) {
++my_concurrency;
task * new_task = create_body_task(*(op->elem));
op->bypass_t = new_task;
__TBB_store_with_release(op->status, SUCCEEDED);
} else if ( my_queue && my_queue->push(*(op->elem)) ) {
op->bypass_t = SUCCESSFULLY_ENQUEUED;
__TBB_store_with_release(op->status, SUCCEEDED);
} else {
op->bypass_t = NULL;
__TBB_store_with_release(op->status, FAILED);
}
}
//! Tries to spawn bodies if available and if concurrency allows
void internal_forward(my_operation *op) {
op->bypass_t = NULL;
if (my_concurrency<my_max_concurrency || !my_max_concurrency) {
input_type i;
bool item_was_retrieved = false;
if ( my_queue )
item_was_retrieved = my_queue->pop(i);
else
item_was_retrieved = my_predecessors.get_item(i);
if (item_was_retrieved) {
++my_concurrency;
op->bypass_t = create_body_task(i);
__TBB_store_with_release(op->status, SUCCEEDED);
return;
}
}
__TBB_store_with_release(op->status, FAILED);
forwarder_busy = false;
}
//! Applies the body to the provided input
// then decides if more work is available
void apply_body( input_type &i ) {
task *new_task = apply_body_bypass(i);
if(!new_task) return;
if(new_task == SUCCESSFULLY_ENQUEUED) return;
FLOW_SPAWN(*new_task);
return;
}
//! Applies the body to the provided input
// then decides if more work is available
task * apply_body_bypass( input_type &i ) {
task * new_task = static_cast<ImplType *>(this)->apply_body_impl_bypass(i);
if ( my_max_concurrency != 0 ) {
my_operation op_data(app_body_bypass); // tries to pop an item or get_item, enqueues another apply_body
my_aggregator.execute(&op_data);
tbb::task *ttask = op_data.bypass_t;
new_task = combine_tasks(new_task, ttask);
}
return new_task;
}
//! allocates a task to call apply_body( input )
inline task * create_body_task( const input_type &input ) {
task* tp = my_graph.root_task();
return (tp) ?
new(task::allocate_additional_child_of(*tp))
apply_body_task_bypass < my_class, input_type >(*this, input) :
NULL;
}
//! Spawns a task that calls apply_body( input )
inline void spawn_body_task( const input_type &input ) {
task* tp = create_body_task(input);
// tp == NULL => g.reset(), which shouldn't occur in concurrent context
if(tp) {
FLOW_SPAWN(*tp);
}
}
//! This is executed by an enqueued task, the "forwarder"
task *forward_task() {
my_operation op_data(try_fwd);
task *rval = NULL;
do {
op_data.status = WAIT;
my_aggregator.execute(&op_data);
if(op_data.status == SUCCEEDED) {
tbb::task *ttask = op_data.bypass_t;
rval = combine_tasks(rval, ttask);
}
} while (op_data.status == SUCCEEDED);
return rval;
}
inline task *create_forward_task() {
task* tp = my_graph.root_task();
return (tp) ?
new(task::allocate_additional_child_of(*tp)) forward_task_bypass< my_class >(*this) :
NULL;
}
//! Spawns a task that calls forward()
inline void spawn_forward_task() {
task* tp = create_forward_task();
if(tp) {
FLOW_SPAWN(*tp);
}
}
}; // function_input_base
//! Implements methods for a function node that takes a type Input as input and sends
// a type Output to its successors.
template< typename Input, typename Output, typename A>
class function_input : public function_input_base<Input, A, function_input<Input,Output,A> > {
public:
typedef Input input_type;
typedef Output output_type;
typedef function_input<Input,Output,A> my_class;
typedef function_input_base<Input, A, my_class> base_type;
typedef function_input_queue<input_type, A> input_queue_type;
// constructor
template<typename Body>
function_input( graph &g, size_t max_concurrency, Body& body, function_input_queue<input_type,A> *q = NULL ) :
base_type(g, max_concurrency, q),
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) {
}
//! Copy constructor
function_input( const function_input& src, input_queue_type *q = NULL ) :
base_type(src, q),
my_body( src.my_body->clone() ) {
}
~function_input() {
delete my_body;
}
template< typename Body >
Body copy_function_object() {
internal::function_body<input_type, output_type> &body_ref = *this->my_body;
return dynamic_cast< internal::function_body_leaf<input_type, output_type, Body> & >(body_ref).get_body();
}
task * apply_body_impl_bypass( const input_type &i) {
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
// There is an extra copied needed to capture the
// body execution without the try_put
tbb::internal::fgt_begin_body( my_body );
output_type v = (*my_body)(i);
tbb::internal::fgt_end_body( my_body );
task * new_task = successors().try_put_task( v );
#else
task * new_task = successors().try_put_task( (*my_body)(i) );
#endif
return new_task;
}
protected:
void reset_function_input(__TBB_PFG_RESET_ARG(reset_flags f)) {
base_type::reset_function_input_base(__TBB_PFG_RESET_ARG(f));
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
if(f & rf_reset_bodies) my_body->reset_body();
#endif
}
function_body<input_type, output_type> *my_body;
virtual broadcast_cache<output_type > &successors() = 0;
}; // function_input
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
// helper templates to reset the successor edges of the output ports of an multifunction_node
template<int N>
struct reset_element {
template<typename P>
static void reset_this(P &p, reset_flags f) {
(void)tbb::flow::get<N-1>(p).successors().reset(f);
reset_element<N-1>::reset_this(p, f);
}
template<typename P>
static bool this_empty(P &p) {
if(tbb::flow::get<N-1>(p).successors().empty())
return reset_element<N-1>::this_empty(p);
return false;
}
};
template<>
struct reset_element<1> {
template<typename P>
static void reset_this(P &p, reset_flags f) {
(void)tbb::flow::get<0>(p).successors().reset(f);
}
template<typename P>
static bool this_empty(P &p) {
return tbb::flow::get<0>(p).successors().empty();
}
};
#endif
//! Implements methods for a function node that takes a type Input as input
// and has a tuple of output ports specified.
template< typename Input, typename OutputPortSet, typename A>
class multifunction_input : public function_input_base<Input, A, multifunction_input<Input,OutputPortSet,A> > {
public:
static const int N = tbb::flow::tuple_size<OutputPortSet>::value;
typedef Input input_type;
typedef OutputPortSet output_ports_type;
typedef multifunction_input<Input,OutputPortSet,A> my_class;
typedef function_input_base<Input, A, my_class> base_type;
typedef function_input_queue<input_type, A> input_queue_type;
// constructor
template<typename Body>
multifunction_input(
graph &g,
size_t max_concurrency,
Body& body,
function_input_queue<input_type,A> *q = NULL ) :
base_type(g, max_concurrency, q),
my_body( new internal::multifunction_body_leaf<input_type, output_ports_type, Body>(body) ) {
}
//! Copy constructor
multifunction_input( const multifunction_input& src, input_queue_type *q = NULL ) :
base_type(src, q),
my_body( src.my_body->clone() ) {
}
~multifunction_input() {
delete my_body;
}
template< typename Body >
Body copy_function_object() {
internal::multifunction_body<input_type, output_ports_type> &body_ref = *this->my_body;
return dynamic_cast< internal::multifunction_body_leaf<input_type, output_ports_type, Body> & >(body_ref).get_body();
}
// for multifunction nodes we do not have a single successor as such. So we just tell
// the task we were successful.
task * apply_body_impl_bypass( const input_type &i) {
tbb::internal::fgt_begin_body( my_body );
(*my_body)(i, my_output_ports);
tbb::internal::fgt_end_body( my_body );
task * new_task = SUCCESSFULLY_ENQUEUED;
return new_task;
}
output_ports_type &output_ports(){ return my_output_ports; }
protected:
/*override*/void reset(__TBB_PFG_RESET_ARG(reset_flags f)) {
base_type::reset_function_input_base(__TBB_PFG_RESET_ARG(f));
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
reset_element<N>::reset_this(my_output_ports, f);
if(f & rf_reset_bodies) my_body->reset_body();
__TBB_ASSERT(!(f & rf_extract) || reset_element<N>::this_empty(my_output_ports), "multifunction_node reset failed");
#endif
}
multifunction_body<input_type, output_ports_type> *my_body;
output_ports_type my_output_ports;
}; // multifunction_input
// template to refer to an output port of a multifunction_node
template<size_t N, typename MOP>
typename tbb::flow::tuple_element<N, typename MOP::output_ports_type>::type &output_port(MOP &op) {
return tbb::flow::get<N>(op.output_ports());
}
// helper structs for split_node
template<int N>
struct emit_element {
template<typename T, typename P>
static void emit_this(const T &t, P &p) {
(void)tbb::flow::get<N-1>(p).try_put(tbb::flow::get<N-1>(t));
emit_element<N-1>::emit_this(t,p);
}
};
template<>
struct emit_element<1> {
template<typename T, typename P>
static void emit_this(const T &t, P &p) {
(void)tbb::flow::get<0>(p).try_put(tbb::flow::get<0>(t));
}
};
//! Implements methods for an executable node that takes continue_msg as input
template< typename Output >
class continue_input : public continue_receiver {
public:
//! The input type of this receiver
typedef continue_msg input_type;
//! The output type of this receiver
typedef Output output_type;
template< typename Body >
continue_input( graph &g, Body& body )
: my_graph_ptr(&g),
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) { }
template< typename Body >
continue_input( graph &g, int number_of_predecessors, Body& body )
: continue_receiver( number_of_predecessors ), my_graph_ptr(&g),
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) { }
continue_input( const continue_input& src ) : continue_receiver(src),
my_graph_ptr(src.my_graph_ptr), my_body( src.my_body->clone() ) {}
~continue_input() {
delete my_body;
}
template< typename Body >
Body copy_function_object() {
internal::function_body<input_type, output_type> &body_ref = *my_body;
return dynamic_cast< internal::function_body_leaf<input_type, output_type, Body> & >(body_ref).get_body();
}
/*override*/void reset_receiver( __TBB_PFG_RESET_ARG(reset_flags f)) {
continue_receiver::reset_receiver(__TBB_PFG_RESET_ARG(f));
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
if(f & rf_reset_bodies) my_body->reset_body();
#endif
}
protected:
graph* my_graph_ptr;
function_body<input_type, output_type> *my_body;
virtual broadcast_cache<output_type > &successors() = 0;
friend class apply_body_task_bypass< continue_input< Output >, continue_msg >;
//! Applies the body to the provided input
/* override */ task *apply_body_bypass( input_type ) {
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
// There is an extra copied needed to capture the
// body execution without the try_put
tbb::internal::fgt_begin_body( my_body );
output_type v = (*my_body)( continue_msg() );
tbb::internal::fgt_end_body( my_body );
return successors().try_put_task( v );
#else
return successors().try_put_task( (*my_body)( continue_msg() ) );
#endif
}
//! Spawns a task that applies the body
/* override */ task *execute( ) {
task* tp = my_graph_ptr->root_task();
return (tp) ?
new ( task::allocate_additional_child_of( *tp ) )
apply_body_task_bypass< continue_input< Output >, continue_msg >( *this, continue_msg() ) :
NULL;
}
}; // continue_input
//! Implements methods for both executable and function nodes that puts Output to its successors
template< typename Output >
class function_output : public sender<Output> {
public:
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
template<int N> friend struct reset_element;
#endif
typedef Output output_type;
typedef receiver<output_type> successor_type;
typedef broadcast_cache<output_type> broadcast_cache_type;
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
typedef std::vector<successor_type *> successor_vector_type;
#endif
function_output() { my_successors.set_owner(this); }
function_output(const function_output & /*other*/) : sender<output_type>() {
my_successors.set_owner(this);
}
//! Adds a new successor to this node
/* override */ bool register_successor( receiver<output_type> &r ) {
successors().register_successor( r );
return true;
}
//! Removes a successor from this node
/* override */ bool remove_successor( receiver<output_type> &r ) {
successors().remove_successor( r );
return true;
}
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
/*override*/ void internal_add_built_successor( receiver<output_type> &r) {
successors().internal_add_built_successor( r );
}
/*override*/ void internal_delete_built_successor( receiver<output_type> &r) {
successors().internal_delete_built_successor( r );
}
/*override*/ size_t successor_count() {
return successors().successor_count();
}
/*override*/ void copy_successors( successor_vector_type &v) {
successors().copy_successors(v);
}
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
// for multifunction_node. The function_body that implements
// the node will have an input and an output tuple of ports. To put
// an item to a successor, the body should
//
// get<I>(output_ports).try_put(output_value);
//
// return value will be bool returned from successors.try_put.
task *try_put_task(const output_type &i) { return my_successors.try_put_task(i); }
protected:
broadcast_cache_type my_successors;
broadcast_cache_type &successors() { return my_successors; }
}; // function_output
template< typename Output >
class multifunction_output : public function_output<Output> {
public:
typedef Output output_type;
typedef function_output<output_type> base_type;
using base_type::my_successors;
multifunction_output() : base_type() {my_successors.set_owner(this);}
multifunction_output( const multifunction_output &/*other*/) : base_type() { my_successors.set_owner(this); }
bool try_put(const output_type &i) {
task *res = my_successors.try_put_task(i);
if(!res) return false;
if(res != SUCCESSFULLY_ENQUEUED) FLOW_SPAWN(*res);
return true;
}
}; // multifunction_output
} // internal
#endif // __TBB__flow_graph_node_impl_H

View File

@@ -0,0 +1,251 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
// tagged buffer that can expand, and can support as many deletions as additions
// list-based, with elements of list held in array (for destruction management),
// multiplicative hashing (like ets). No synchronization built-in.
//
#ifndef __TBB__flow_graph_tagged_buffer_impl_H
#define __TBB__flow_graph_tagged_buffer_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
// included in namespace tbb::flow::interface7::internal
template<typename T, typename U, size_t NoTagMark>
struct otherData {
T t;
U next;
otherData() : t(NoTagMark), next(NULL) {}
};
template<typename TagType, typename ValueType, size_t NoTagMark>
struct buffer_element_type {
// the second parameter below is void * because we can't forward-declare the type
// itself, so we just reinterpret_cast below.
typedef typename aligned_pair<ValueType, otherData<TagType, void *, NoTagMark> >::type type;
};
template
<
typename TagType,
typename ValueType,
size_t NoTagMark = 0,
typename Allocator=tbb::cache_aligned_allocator< typename buffer_element_type<TagType, ValueType, NoTagMark>::type >
>
class tagged_buffer {
public:
static const size_t INITIAL_SIZE = 8; // initial size of the hash pointer table
static const TagType NO_TAG = TagType(NoTagMark);
typedef ValueType value_type;
typedef typename buffer_element_type<TagType, ValueType, NO_TAG>::type element_type;
typedef value_type *pointer_type;
typedef element_type *list_array_type; // array we manage manually
typedef list_array_type *pointer_array_type;
typedef typename Allocator::template rebind<list_array_type>::other pointer_array_allocator_type;
typedef typename Allocator::template rebind<element_type>::other elements_array_allocator;
private:
size_t my_size;
size_t nelements;
pointer_array_type pointer_array; // pointer_array[my_size]
list_array_type elements_array; // elements_array[my_size / 2]
element_type* free_list;
size_t mask() { return my_size - 1; }
static size_t hash(TagType t) {
return uintptr_t(t)*tbb::internal::select_size_t_constant<0x9E3779B9,0x9E3779B97F4A7C15ULL>::value;
}
void set_up_free_list( element_type **p_free_list, list_array_type la, size_t sz) {
for(size_t i=0; i < sz - 1; ++i ) { // construct free list
la[i].second.next = &(la[i+1]);
la[i].second.t = NO_TAG;
}
la[sz-1].second.next = NULL;
*p_free_list = &(la[0]);
}
// cleanup for exceptions
struct DoCleanup {
pointer_array_type *my_pa;
list_array_type *my_elements;
size_t my_size;
DoCleanup(pointer_array_type &pa, list_array_type &my_els, size_t sz) :
my_pa(&pa), my_elements(&my_els), my_size(sz) { }
~DoCleanup() {
if(my_pa) {
size_t dont_care = 0;
internal_free_buffer(*my_pa, *my_elements, my_size, dont_care);
}
}
};
// exception-safety requires we do all the potentially-throwing operations first
void grow_array() {
size_t new_size = my_size*2;
size_t new_nelements = nelements; // internal_free_buffer zeroes this
list_array_type new_elements_array = NULL;
pointer_array_type new_pointer_array = NULL;
list_array_type new_free_list = NULL;
{
DoCleanup my_cleanup(new_pointer_array, new_elements_array, new_size);
new_elements_array = elements_array_allocator().allocate(my_size);
new_pointer_array = pointer_array_allocator_type().allocate(new_size);
for(size_t i=0; i < new_size; ++i) new_pointer_array[i] = NULL;
set_up_free_list(&new_free_list, new_elements_array, my_size );
for(size_t i=0; i < my_size; ++i) {
for( element_type* op = pointer_array[i]; op; op = (element_type *)(op->second.next)) {
value_type *ov = reinterpret_cast<value_type *>(&(op->first));
// could have std::move semantics
internal_tagged_insert(new_pointer_array, new_size, new_free_list, op->second.t, *ov);
}
}
my_cleanup.my_pa = NULL;
my_cleanup.my_elements = NULL;
}
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
free_list = new_free_list;
pointer_array = new_pointer_array;
elements_array = new_elements_array;
my_size = new_size;
nelements = new_nelements;
}
// v should have perfect forwarding if std::move implemented.
// we use this method to move elements in grow_array, so can't use class fields
void internal_tagged_insert( element_type **p_pointer_array, size_t p_sz, list_array_type &p_free_list,
const TagType t, const value_type &v) {
size_t l_mask = p_sz-1;
size_t h = hash(t) & l_mask;
__TBB_ASSERT(p_free_list, "Error: free list not set up.");
element_type* my_elem = p_free_list; p_free_list = (element_type *)(p_free_list->second.next);
my_elem->second.t = t;
(void) new(&(my_elem->first)) value_type(v);
my_elem->second.next = p_pointer_array[h];
p_pointer_array[h] = my_elem;
}
void internal_initialize_buffer() {
pointer_array = pointer_array_allocator_type().allocate(my_size);
for(size_t i = 0; i < my_size; ++i) pointer_array[i] = NULL;
elements_array = elements_array_allocator().allocate(my_size / 2);
set_up_free_list(&free_list, elements_array, my_size / 2);
}
// made static so an enclosed class can use to properly dispose of the internals
static void internal_free_buffer( pointer_array_type &pa, list_array_type &el, size_t &sz, size_t &ne ) {
if(pa) {
for(size_t i = 0; i < sz; ++i ) {
element_type *p_next;
for( element_type *p = pa[i]; p; p = p_next) {
p_next = (element_type *)p->second.next;
value_type *vp = reinterpret_cast<value_type *>(&(p->first));
vp->~value_type();
}
}
pointer_array_allocator_type().deallocate(pa, sz);
pa = NULL;
}
// Separate test (if allocation of pa throws, el may be allocated.
// but no elements will be constructed.)
if(el) {
elements_array_allocator().deallocate(el, sz / 2);
el = NULL;
}
sz = INITIAL_SIZE;
ne = 0;
}
public:
tagged_buffer() : my_size(INITIAL_SIZE), nelements(0) {
internal_initialize_buffer();
}
~tagged_buffer() {
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
}
void reset() {
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
internal_initialize_buffer();
}
bool tagged_insert(const TagType t, const value_type &v) {
pointer_type p;
if(tagged_find_ref(t, p)) {
p->~value_type();
(void) new(p) value_type(v); // copy-construct into the space
return false;
}
++nelements;
if(nelements*2 > my_size) grow_array();
internal_tagged_insert(pointer_array, my_size, free_list, t, v);
return true;
}
// returns reference to array element.v
bool tagged_find_ref(const TagType t, pointer_type &v) {
size_t i = hash(t) & mask();
for(element_type* p = pointer_array[i]; p; p = (element_type *)(p->second.next)) {
if(p->second.t == t) {
v = reinterpret_cast<pointer_type>(&(p->first));
return true;
}
}
return false;
}
bool tagged_find( const TagType t, value_type &v) {
value_type *p;
if(tagged_find_ref(t, p)) {
v = *p;
return true;
}
else
return false;
}
void tagged_delete(const TagType t) {
size_t h = hash(t) & mask();
element_type* prev = NULL;
for(element_type* p = pointer_array[h]; p; prev = p, p = (element_type *)(p->second.next)) {
if(p->second.t == t) {
value_type *vp = reinterpret_cast<value_type *>(&(p->first));
vp->~value_type();
p->second.t = NO_TAG;
if(prev) prev->second.next = p->second.next;
else pointer_array[h] = (element_type *)(p->second.next);
p->second.next = free_list;
free_list = p;
--nelements;
return;
}
}
__TBB_ASSERT(false, "tag not found for delete");
}
};
#endif // __TBB__flow_graph_tagged_buffer_impl_H

View File

@@ -0,0 +1,205 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _FGT_GRAPH_TRACE_IMPL_H
#define _FGT_GRAPH_TRACE_IMPL_H
#include "../tbb_profiling.h"
namespace tbb {
namespace internal {
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
static inline void fgt_internal_create_input_port( void *node, void *p, string_index name_index ) {
itt_make_task_group( ITT_DOMAIN_FLOW, p, FLOW_INPUT_PORT, node, FLOW_NODE, name_index );
}
static inline void fgt_internal_create_output_port( void *node, void *p, string_index name_index ) {
itt_make_task_group( ITT_DOMAIN_FLOW, p, FLOW_OUTPUT_PORT, node, FLOW_NODE, name_index );
}
template < typename TypesTuple, typename PortsTuple, int N >
struct fgt_internal_input_helper {
static void register_port( void *node, PortsTuple &ports ) {
fgt_internal_create_input_port( node, (void*)static_cast< tbb::flow::interface7::receiver< typename tbb::flow::tuple_element<N-1,TypesTuple>::type > * >(&(tbb::flow::get<N-1>(ports))),
static_cast<tbb::internal::string_index>(FLOW_INPUT_PORT_0 + N - 1) );
fgt_internal_input_helper<TypesTuple, PortsTuple, N-1>::register_port( node, ports );
}
};
template < typename TypesTuple, typename PortsTuple >
struct fgt_internal_input_helper<TypesTuple,PortsTuple,1> {
static void register_port( void *node, PortsTuple &ports ) {
fgt_internal_create_input_port( node, (void*)static_cast< tbb::flow::interface7::receiver< typename tbb::flow::tuple_element<0,TypesTuple>::type > * >(&(tbb::flow::get<0>(ports))),
FLOW_INPUT_PORT_0 );
}
};
template < typename TypesTuple, typename PortsTuple, int N >
struct fgt_internal_output_helper {
static void register_port( void *node, PortsTuple &ports ) {
fgt_internal_create_output_port( node, (void*)static_cast< tbb::flow::interface7::sender< typename tbb::flow::tuple_element<N-1,TypesTuple>::type > * >(&(tbb::flow::get<N-1>(ports))),
static_cast<tbb::internal::string_index>(FLOW_OUTPUT_PORT_0 + N - 1) );
fgt_internal_output_helper<TypesTuple, PortsTuple, N-1>::register_port( node, ports );
}
};
template < typename TypesTuple, typename PortsTuple >
struct fgt_internal_output_helper<TypesTuple,PortsTuple,1> {
static void register_port( void *node, PortsTuple &ports ) {
fgt_internal_create_output_port( node, (void*)static_cast< tbb::flow::interface7::sender< typename tbb::flow::tuple_element<0,TypesTuple>::type > * >(&(tbb::flow::get<0>(ports))),
FLOW_OUTPUT_PORT_0 );
}
};
template< typename NodeType >
void fgt_multioutput_node_desc( const NodeType *node, const char *desc ) {
void *addr = (void *)( static_cast< tbb::flow::interface7::receiver< typename NodeType::input_type > * >(const_cast< NodeType *>(node)) );
itt_metadata_str_add( ITT_DOMAIN_FLOW, addr, FLOW_NODE, FLOW_OBJECT_NAME, desc );
}
template< typename NodeType >
static inline void fgt_node_desc( const NodeType *node, const char *desc ) {
void *addr = (void *)( static_cast< tbb::flow::interface7::sender< typename NodeType::output_type > * >(const_cast< NodeType *>(node)) );
itt_metadata_str_add( ITT_DOMAIN_FLOW, addr, FLOW_NODE, FLOW_OBJECT_NAME, desc );
}
static inline void fgt_graph_desc( void *g, const char *desc ) {
itt_metadata_str_add( ITT_DOMAIN_FLOW, g, FLOW_GRAPH, FLOW_OBJECT_NAME, desc );
}
static inline void fgt_body( void *node, void *body ) {
itt_relation_add( ITT_DOMAIN_FLOW, body, FLOW_BODY, __itt_relation_is_child_of, node, FLOW_NODE );
}
template< typename OutputTuple, int N, typename PortsTuple >
static inline void fgt_multioutput_node( string_index t, void *g, void *input_port, PortsTuple &ports ) {
itt_make_task_group( ITT_DOMAIN_FLOW, input_port, FLOW_NODE, g, FLOW_GRAPH, t );
fgt_internal_create_input_port( input_port, input_port, FLOW_INPUT_PORT_0 );
fgt_internal_output_helper<OutputTuple, PortsTuple, N>::register_port( input_port, ports );
}
template< typename OutputTuple, int N, typename PortsTuple >
static inline void fgt_multioutput_node_with_body( string_index t, void *g, void *input_port, PortsTuple &ports, void *body ) {
itt_make_task_group( ITT_DOMAIN_FLOW, input_port, FLOW_NODE, g, FLOW_GRAPH, t );
fgt_internal_create_input_port( input_port, input_port, FLOW_INPUT_PORT_0 );
fgt_internal_output_helper<OutputTuple, PortsTuple, N>::register_port( input_port, ports );
fgt_body( input_port, body );
}
template< typename InputTuple, int N, typename PortsTuple >
static inline void fgt_multiinput_node( string_index t, void *g, PortsTuple &ports, void *output_port) {
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
fgt_internal_input_helper<InputTuple, PortsTuple, N>::register_port( output_port, ports );
}
static inline void fgt_node( string_index t, void *g, void *output_port ) {
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
}
static inline void fgt_node_with_body( string_index t, void *g, void *output_port, void *body ) {
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
fgt_body( output_port, body );
}
static inline void fgt_node( string_index t, void *g, void *input_port, void *output_port ) {
fgt_node( t, g, output_port );
fgt_internal_create_input_port( output_port, input_port, FLOW_INPUT_PORT_0 );
}
static inline void fgt_node_with_body( string_index t, void *g, void *input_port, void *output_port, void *body ) {
fgt_node_with_body( t, g, output_port, body );
fgt_internal_create_input_port( output_port, input_port, FLOW_INPUT_PORT_0 );
}
static inline void fgt_node( string_index t, void *g, void *input_port, void *decrement_port, void *output_port ) {
fgt_node( t, g, input_port, output_port );
fgt_internal_create_input_port( output_port, decrement_port, FLOW_INPUT_PORT_1 );
}
static inline void fgt_make_edge( void *output_port, void *input_port ) {
itt_relation_add( ITT_DOMAIN_FLOW, output_port, FLOW_OUTPUT_PORT, __itt_relation_is_predecessor_to, input_port, FLOW_INPUT_PORT);
}
static inline void fgt_remove_edge( void *output_port, void *input_port ) {
itt_relation_add( ITT_DOMAIN_FLOW, output_port, FLOW_OUTPUT_PORT, __itt_relation_is_sibling_of, input_port, FLOW_INPUT_PORT);
}
static inline void fgt_graph( void *g ) {
itt_make_task_group( ITT_DOMAIN_FLOW, g, FLOW_GRAPH, NULL, FLOW_NULL, FLOW_GRAPH );
}
static inline void fgt_begin_body( void *body ) {
itt_task_begin( ITT_DOMAIN_FLOW, body, FLOW_BODY, NULL, FLOW_NULL, FLOW_NULL );
}
static inline void fgt_end_body( void * ) {
itt_task_end( ITT_DOMAIN_FLOW );
}
#else // TBB_PREVIEW_FLOW_GRAPH_TRACE
static inline void fgt_graph( void * /*g*/ ) { }
template< typename NodeType >
static inline void fgt_multioutput_node_desc( const NodeType * /*node*/, const char * /*desc*/ ) { }
template< typename NodeType >
static inline void fgt_node_desc( const NodeType * /*node*/, const char * /*desc*/ ) { }
static inline void fgt_graph_desc( void * /*g*/, const char * /*desc*/ ) { }
static inline void fgt_body( void * /*node*/, void * /*body*/ ) { }
template< typename OutputTuple, int N, typename PortsTuple >
static inline void fgt_multioutput_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, PortsTuple & /*ports*/ ) { }
template< typename OutputTuple, int N, typename PortsTuple >
static inline void fgt_multioutput_node_with_body( string_index /*t*/, void * /*g*/, void * /*input_port*/, PortsTuple & /*ports*/, void * /*body*/ ) { }
template< typename InputTuple, int N, typename PortsTuple >
static inline void fgt_multiinput_node( string_index /*t*/, void * /*g*/, PortsTuple & /*ports*/, void * /*output_port*/ ) { }
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*output_port*/ ) { }
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*output_port*/ ) { }
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*decrement_port*/, void * /*output_port*/ ) { }
static inline void fgt_node_with_body( string_index /*t*/, void * /*g*/, void * /*output_port*/, void * /*body*/ ) { }
static inline void fgt_node_with_body( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*output_port*/, void * /*body*/ ) { }
static inline void fgt_make_edge( void * /*output_port*/, void * /*input_port*/ ) { }
static inline void fgt_remove_edge( void * /*output_port*/, void * /*input_port*/ ) { }
static inline void fgt_begin_body( void * /*body*/ ) { }
static inline void fgt_end_body( void * /*body*/) { }
#endif // TBB_PREVIEW_FLOW_GRAPH_TRACE
} // namespace internal
} // namespace tbb
#endif

View File

@@ -0,0 +1,497 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__flow_graph_types_impl_H
#define __TBB__flow_graph_types_impl_H
#ifndef __TBB_flow_graph_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
// included in namespace tbb::flow::interface7
namespace internal {
// wrap each element of a tuple in a template, and make a tuple of the result.
template<int N, template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements;
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<1, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type> >
type;
};
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<2, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type> >
type;
};
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<3, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type> >
type;
};
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<4, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type> >
type;
};
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<5, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type> >
type;
};
#if __TBB_VARIADIC_MAX >= 6
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<6, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type> >
type;
};
#endif
#if __TBB_VARIADIC_MAX >= 7
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<7, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type> >
type;
};
#endif
#if __TBB_VARIADIC_MAX >= 8
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<8, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type> >
type;
};
#endif
#if __TBB_VARIADIC_MAX >= 9
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<9, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<8,TypeTuple>::type> >
type;
};
#endif
#if __TBB_VARIADIC_MAX >= 10
template<template<class> class PT, typename TypeTuple>
struct wrap_tuple_elements<10, PT, TypeTuple> {
typedef typename tbb::flow::tuple<
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<8,TypeTuple>::type>,
PT<typename tbb::flow::tuple_element<9,TypeTuple>::type> >
type;
};
#endif
//! type mimicking std::pair but with trailing fill to ensure each element of an array
//* will have the correct alignment
template<typename T1, typename T2, size_t REM>
struct type_plus_align {
char first[sizeof(T1)];
T2 second;
char fill1[REM];
};
template<typename T1, typename T2>
struct type_plus_align<T1,T2,0> {
char first[sizeof(T1)];
T2 second;
};
template<class U> struct alignment_of {
typedef struct { char t; U padded; } test_alignment;
static const size_t value = sizeof(test_alignment) - sizeof(U);
};
// T1, T2 are actual types stored. The space defined for T1 in the type returned
// is a char array of the correct size. Type T2 should be trivially-constructible,
// T1 must be explicitly managed.
template<typename T1, typename T2>
struct aligned_pair {
static const size_t t1_align = alignment_of<T1>::value;
static const size_t t2_align = alignment_of<T2>::value;
typedef type_plus_align<T1, T2, 0 > just_pair;
static const size_t max_align = t1_align < t2_align ? t2_align : t1_align;
static const size_t extra_bytes = sizeof(just_pair) % max_align;
static const size_t remainder = extra_bytes ? max_align - extra_bytes : 0;
public:
typedef type_plus_align<T1,T2,remainder> type;
}; // aligned_pair
// support for variant type
// type we use when we're not storing a value
struct default_constructed { };
// type which contains another type, tests for what type is contained, and references to it.
// internal::Wrapper<T>
// void CopyTo( void *newSpace) : builds a Wrapper<T> copy of itself in newSpace
// struct to allow us to copy and test the type of objects
struct WrapperBase {
virtual ~WrapperBase() {}
virtual void CopyTo(void* /*newSpace*/) const { }
};
// Wrapper<T> contains a T, with the ability to test what T is. The Wrapper<T> can be
// constructed from a T, can be copy-constructed from another Wrapper<T>, and can be
// examined via value(), but not modified.
template<typename T>
struct Wrapper: public WrapperBase {
typedef T value_type;
typedef T* pointer_type;
private:
T value_space;
public:
const value_type &value() const { return value_space; }
private:
Wrapper();
// on exception will ensure the Wrapper will contain only a trivially-constructed object
struct _unwind_space {
pointer_type space;
_unwind_space(pointer_type p) : space(p) {}
~_unwind_space() {
if(space) (void) new (space) Wrapper<default_constructed>(default_constructed());
}
};
public:
explicit Wrapper( const T& other ) : value_space(other) { }
explicit Wrapper(const Wrapper& other) : value_space(other.value_space) { }
/*override*/void CopyTo(void* newSpace) const {
_unwind_space guard((pointer_type)newSpace);
(void) new(newSpace) Wrapper(value_space);
guard.space = NULL;
}
/*override*/~Wrapper() { }
};
// specialization for array objects
template<typename T, size_t N>
struct Wrapper<T[N]> : public WrapperBase {
typedef T value_type;
typedef T* pointer_type;
// space must be untyped.
typedef T ArrayType[N];
private:
// The space is not of type T[N] because when copy-constructing, it would be
// default-initialized and then copied to in some fashion, resulting in two
// constructions and one destruction per element. If the type is char[ ], we
// placement new into each element, resulting in one construction per element.
static const size_t space_size = sizeof(ArrayType) / sizeof(char);
char value_space[space_size];
// on exception will ensure the already-built objects will be destructed
// (the value_space is a char array, so it is already trivially-destructible.)
struct _unwind_class {
pointer_type space;
int already_built;
_unwind_class(pointer_type p) : space(p), already_built(0) {}
~_unwind_class() {
if(space) {
for(size_t i = already_built; i > 0 ; --i ) space[i-1].~value_type();
(void) new(space) Wrapper<default_constructed>(default_constructed());
}
}
};
public:
const ArrayType &value() const {
char *vp = const_cast<char *>(value_space);
return reinterpret_cast<ArrayType &>(*vp);
}
private:
Wrapper();
public:
// have to explicitly construct because other decays to a const value_type*
explicit Wrapper(const ArrayType& other) {
_unwind_class guard((pointer_type)value_space);
pointer_type vp = reinterpret_cast<pointer_type>(&value_space);
for(size_t i = 0; i < N; ++i ) {
(void) new(vp++) value_type(other[i]);
++(guard.already_built);
}
guard.space = NULL;
}
explicit Wrapper(const Wrapper& other) : WrapperBase() {
// we have to do the heavy lifting to copy contents
_unwind_class guard((pointer_type)value_space);
pointer_type dp = reinterpret_cast<pointer_type>(value_space);
pointer_type sp = reinterpret_cast<pointer_type>(const_cast<char *>(other.value_space));
for(size_t i = 0; i < N; ++i, ++dp, ++sp) {
(void) new(dp) value_type(*sp);
++(guard.already_built);
}
guard.space = NULL;
}
/*override*/void CopyTo(void* newSpace) const {
(void) new(newSpace) Wrapper(*this); // exceptions handled in copy constructor
}
/*override*/~Wrapper() {
// have to destroy explicitly in reverse order
pointer_type vp = reinterpret_cast<pointer_type>(&value_space);
for(size_t i = N; i > 0 ; --i ) vp[i-1].~value_type();
}
};
// given a tuple, return the type of the element that has the maximum alignment requirement.
// Given a tuple and that type, return the number of elements of the object with the max
// alignment requirement that is at least as big as the largest object in the tuple.
template<bool, class T1, class T2> struct pick_one;
template<class T1, class T2> struct pick_one<true , T1, T2> { typedef T1 type; };
template<class T1, class T2> struct pick_one<false, T1, T2> { typedef T2 type; };
template< template<class> class Selector, typename T1, typename T2 >
struct pick_max {
typedef typename pick_one< (Selector<T1>::value > Selector<T2>::value), T1, T2 >::type type;
};
template<typename T> struct size_of { static const int value = sizeof(T); };
template< size_t N, class Tuple, template<class> class Selector > struct pick_tuple_max {
typedef typename pick_tuple_max<N-1, Tuple, Selector>::type LeftMaxType;
typedef typename tbb::flow::tuple_element<N-1, Tuple>::type ThisType;
typedef typename pick_max<Selector, LeftMaxType, ThisType>::type type;
};
template< class Tuple, template<class> class Selector > struct pick_tuple_max<0, Tuple, Selector> {
typedef typename tbb::flow::tuple_element<0, Tuple>::type type;
};
// is the specified type included in a tuple?
template<class U, class V> struct is_same_type { static const bool value = false; };
template<class W> struct is_same_type<W,W> { static const bool value = true; };
template<class Q, size_t N, class Tuple>
struct is_element_of {
typedef typename tbb::flow::tuple_element<N-1, Tuple>::type T_i;
static const bool value = is_same_type<Q,T_i>::value || is_element_of<Q,N-1,Tuple>::value;
};
template<class Q, class Tuple>
struct is_element_of<Q,0,Tuple> {
typedef typename tbb::flow::tuple_element<0, Tuple>::type T_i;
static const bool value = is_same_type<Q,T_i>::value;
};
// allow the construction of types that are listed tuple. If a disallowed type
// construction is written, a method involving this type is created. The
// type has no definition, so a syntax error is generated.
template<typename T> struct ERROR_Type_Not_allowed_In_Tagged_Msg_Not_Member_Of_Tuple;
template<typename T, bool BUILD_IT> struct do_if;
template<typename T>
struct do_if<T, true> {
static void construct(void *mySpace, const T& x) {
(void) new(mySpace) Wrapper<T>(x);
}
};
template<typename T>
struct do_if<T, false> {
static void construct(void * /*mySpace*/, const T& x) {
// This method is instantiated when the type T does not match any of the
// element types in the Tuple in variant<Tuple>.
ERROR_Type_Not_allowed_In_Tagged_Msg_Not_Member_Of_Tuple<T>::bad_type(x);
}
};
// Tuple tells us the allowed types that variant can hold. It determines the alignment of the space in
// Wrapper, and how big Wrapper is.
//
// the object can only be tested for type, and a read-only reference can be fetched by cast_to<T>().
using tbb::internal::punned_cast;
struct tagged_null_type {};
template<typename TagType, typename T0, typename T1=tagged_null_type, typename T2=tagged_null_type, typename T3=tagged_null_type,
typename T4=tagged_null_type, typename T5=tagged_null_type, typename T6=tagged_null_type,
typename T7=tagged_null_type, typename T8=tagged_null_type, typename T9=tagged_null_type>
class tagged_msg {
typedef tbb::flow::tuple<T0, T1, T2, T3, T4
#if __TBB_VARIADIC_MAX >= 6
, T5
#endif
#if __TBB_VARIADIC_MAX >= 7
, T6
#endif
#if __TBB_VARIADIC_MAX >= 8
, T7
#endif
#if __TBB_VARIADIC_MAX >= 9
, T8
#endif
#if __TBB_VARIADIC_MAX >= 10
, T9
#endif
> Tuple;
private:
class variant {
static const size_t N = tbb::flow::tuple_size<Tuple>::value;
typedef typename pick_tuple_max<N, Tuple, alignment_of>::type AlignType;
typedef typename pick_tuple_max<N, Tuple, size_of>::type MaxSizeType;
static const size_t MaxNBytes = (sizeof(Wrapper<MaxSizeType>)+sizeof(AlignType)-1);
static const size_t MaxNElements = MaxNBytes/sizeof(AlignType);
typedef typename tbb::aligned_space<AlignType, MaxNElements> SpaceType;
SpaceType my_space;
static const size_t MaxSize = sizeof(SpaceType);
public:
variant() { (void) new(&my_space) Wrapper<default_constructed>(default_constructed()); }
template<typename T>
variant( const T& x ) {
do_if<T, is_element_of<T, N, Tuple>::value>::construct(&my_space,x);
}
variant(const variant& other) {
const WrapperBase * h = punned_cast<const WrapperBase *>(&(other.my_space));
h->CopyTo(&my_space);
}
// assignment must destroy and re-create the Wrapper type, as there is no way
// to create a Wrapper-to-Wrapper assign even if we find they agree in type.
void operator=( const variant& rhs ) {
if(&rhs != this) {
WrapperBase *h = punned_cast<WrapperBase *>(&my_space);
h->~WrapperBase();
const WrapperBase *ch = punned_cast<const WrapperBase *>(&(rhs.my_space));
ch->CopyTo(&my_space);
}
}
template<typename U>
const U& variant_cast_to() const {
const Wrapper<U> *h = dynamic_cast<const Wrapper<U>*>(punned_cast<const WrapperBase *>(&my_space));
if(!h) {
tbb::internal::throw_exception(tbb::internal::eid_bad_tagged_msg_cast);
}
return h->value();
}
template<typename U>
bool variant_is_a() const { return dynamic_cast<const Wrapper<U>*>(punned_cast<const WrapperBase *>(&my_space)) != NULL; }
bool variant_is_default_constructed() const {return variant_is_a<default_constructed>();}
~variant() {
WrapperBase *h = punned_cast<WrapperBase *>(&my_space);
h->~WrapperBase();
}
}; //class variant
TagType my_tag;
variant my_msg;
public:
tagged_msg(): my_tag(TagType(~0)), my_msg(){}
template<typename T, typename R>
tagged_msg(T const &index, R const &value) : my_tag(index), my_msg(value) {}
#if __TBB_CONST_REF_TO_ARRAY_TEMPLATE_PARAM_BROKEN
template<typename T, typename R, size_t N>
tagged_msg(T const &index, R (&value)[N]) : my_tag(index), my_msg(value) {}
#endif
void set_tag(TagType const &index) {my_tag = index;}
TagType tag() const {return my_tag;}
template<typename V>
const V& cast_to() const {return my_msg.template variant_cast_to<V>();}
template<typename V>
bool is_a() const {return my_msg.template variant_is_a<V>();}
bool is_default_constructed() const {return my_msg.variant_is_default_constructed();}
}; //class tagged_msg
// template to simplify cast and test for tagged_msg in template contexts
template<typename T, typename V>
const T& cast_to(V const &v) { return v.template cast_to<T>(); }
template<typename T, typename V>
bool is_a(V const &v) { return v.template is_a<T>(); }
} // namespace internal
#endif /* __TBB__flow_graph_types_impl_H */

View File

@@ -0,0 +1,102 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_mutex_padding_H
#define __TBB_mutex_padding_H
// wrapper for padding mutexes to be alone on a cache line, without requiring they be allocated
// from a pool. Because we allow them to be defined anywhere they must be two cache lines in size.
namespace tbb {
namespace interface7 {
namespace internal {
static const size_t cache_line_size = 64;
// Pad a mutex to occupy a number of full cache lines sufficient to avoid false sharing
// with other data; space overhead is up to 2*cache_line_size-1.
template<typename Mutex, bool is_rw> class padded_mutex;
template<typename Mutex>
class padded_mutex<Mutex,false> : tbb::internal::mutex_copy_deprecated_and_disabled {
typedef long pad_type;
pad_type my_pad[((sizeof(Mutex)+cache_line_size-1)/cache_line_size+1)*cache_line_size/sizeof(pad_type)];
Mutex *impl() { return (Mutex *)((uintptr_t(this)|(cache_line_size-1))+1);}
public:
static const bool is_rw_mutex = Mutex::is_rw_mutex;
static const bool is_recursive_mutex = Mutex::is_recursive_mutex;
static const bool is_fair_mutex = Mutex::is_fair_mutex;
padded_mutex() { new(impl()) Mutex(); }
~padded_mutex() { impl()->~Mutex(); }
//! Represents acquisition of a mutex.
class scoped_lock : tbb::internal::no_copy {
typename Mutex::scoped_lock my_scoped_lock;
public:
scoped_lock() : my_scoped_lock() {}
scoped_lock( padded_mutex& m ) : my_scoped_lock(*m.impl()) { }
~scoped_lock() { }
void acquire( padded_mutex& m ) { my_scoped_lock.acquire(*m.impl()); }
bool try_acquire( padded_mutex& m ) { return my_scoped_lock.try_acquire(*m.impl()); }
void release() { my_scoped_lock.release(); }
};
};
template<typename Mutex>
class padded_mutex<Mutex,true> : tbb::internal::mutex_copy_deprecated_and_disabled {
typedef long pad_type;
pad_type my_pad[((sizeof(Mutex)+cache_line_size-1)/cache_line_size+1)*cache_line_size/sizeof(pad_type)];
Mutex *impl() { return (Mutex *)((uintptr_t(this)|(cache_line_size-1))+1);}
public:
static const bool is_rw_mutex = Mutex::is_rw_mutex;
static const bool is_recursive_mutex = Mutex::is_recursive_mutex;
static const bool is_fair_mutex = Mutex::is_fair_mutex;
padded_mutex() { new(impl()) Mutex(); }
~padded_mutex() { impl()->~Mutex(); }
//! Represents acquisition of a mutex.
class scoped_lock : tbb::internal::no_copy {
typename Mutex::scoped_lock my_scoped_lock;
public:
scoped_lock() : my_scoped_lock() {}
scoped_lock( padded_mutex& m, bool write = true ) : my_scoped_lock(*m.impl(),write) { }
~scoped_lock() { }
void acquire( padded_mutex& m, bool write = true ) { my_scoped_lock.acquire(*m.impl(),write); }
bool try_acquire( padded_mutex& m, bool write = true ) { return my_scoped_lock.try_acquire(*m.impl(),write); }
bool upgrade_to_writer() { return my_scoped_lock.upgrade_to_writer(); }
bool downgrade_to_reader() { return my_scoped_lock.downgrade_to_reader(); }
void release() { my_scoped_lock.release(); }
};
};
} // namespace internal
} // namespace interface7
} // namespace tbb
#endif /* __TBB_mutex_padding_H */

View File

@@ -0,0 +1,70 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_range_iterator_H
#define __TBB_range_iterator_H
#include "../tbb_stddef.h"
#if __TBB_CPP11_STD_BEGIN_END_PRESENT && __TBB_CPP11_AUTO_PRESENT && __TBB_CPP11_DECLTYPE_PRESENT
#include <iterator>
#endif
namespace tbb {
// iterators to first and last elements of container
namespace internal {
#if __TBB_CPP11_STD_BEGIN_END_PRESENT && __TBB_CPP11_AUTO_PRESENT && __TBB_CPP11_DECLTYPE_PRESENT
using std::begin;
using std::end;
template<typename Container>
auto first(Container& c)-> decltype(begin(c)) {return begin(c);}
template<typename Container>
auto first(const Container& c)-> decltype(begin(c)) {return begin(c);}
template<typename Container>
auto last(Container& c)-> decltype(begin(c)) {return end(c);}
template<typename Container>
auto last(const Container& c)-> decltype(begin(c)) {return end(c);}
#else
template<typename Container>
typename Container::iterator first(Container& c) {return c.begin();}
template<typename Container>
typename Container::const_iterator first(const Container& c) {return c.begin();}
template<typename Container>
typename Container::iterator last(Container& c) {return c.end();}
template<typename Container>
typename Container::const_iterator last(const Container& c) {return c.end();}
#endif
template<typename T, size_t size>
T* first(T (&arr) [size]) {return arr;}
template<typename T, size_t size>
T* last(T (&arr) [size]) {return arr + size;}
} //namespace internal
} //namespace tbb
#endif // __TBB_range_iterator_H

View File

@@ -0,0 +1,65 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
TBB_STRING_RESOURCE(FLOW_BROADCAST_NODE, "broadcast_node")
TBB_STRING_RESOURCE(FLOW_BUFFER_NODE, "buffer_node")
TBB_STRING_RESOURCE(FLOW_CONTINUE_NODE, "continue_node")
TBB_STRING_RESOURCE(FLOW_FUNCTION_NODE, "function_node")
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_QUEUEING, "join_node (queueing)")
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_RESERVING, "join_node (reserving)")
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_TAG_MATCHING, "join_node (tag_matching)")
TBB_STRING_RESOURCE(FLOW_LIMITER_NODE, "limiter_node")
TBB_STRING_RESOURCE(FLOW_MULTIFUNCTION_NODE, "multifunction_node")
TBB_STRING_RESOURCE(FLOW_OR_NODE, "or_node") //no longer in use, kept for backward compatibilty
TBB_STRING_RESOURCE(FLOW_OVERWRITE_NODE, "overwrite_node")
TBB_STRING_RESOURCE(FLOW_PRIORITY_QUEUE_NODE, "priority_queue_node")
TBB_STRING_RESOURCE(FLOW_QUEUE_NODE, "queue_node")
TBB_STRING_RESOURCE(FLOW_SEQUENCER_NODE, "sequencer_node")
TBB_STRING_RESOURCE(FLOW_SOURCE_NODE, "source_node")
TBB_STRING_RESOURCE(FLOW_SPLIT_NODE, "split_node")
TBB_STRING_RESOURCE(FLOW_WRITE_ONCE_NODE, "write_once_node")
TBB_STRING_RESOURCE(FLOW_BODY, "body")
TBB_STRING_RESOURCE(FLOW_GRAPH, "graph")
TBB_STRING_RESOURCE(FLOW_NODE, "node")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT, "input_port")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_0, "input_port_0")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_1, "input_port_1")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_2, "input_port_2")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_3, "input_port_3")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_4, "input_port_4")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_5, "input_port_5")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_6, "input_port_6")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_7, "input_port_7")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_8, "input_port_8")
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_9, "input_port_9")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT, "output_port")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_0, "output_port_0")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_1, "output_port_1")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_2, "output_port_2")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_3, "output_port_3")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_4, "output_port_4")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_5, "output_port_5")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_6, "output_port_6")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_7, "output_port_7")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_8, "output_port_8")
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_9, "output_port_9")
TBB_STRING_RESOURCE(FLOW_OBJECT_NAME, "object_name")
TBB_STRING_RESOURCE(FLOW_NULL, "null")
TBB_STRING_RESOURCE(FLOW_INDEXER_NODE, "indexer_node")

View File

@@ -0,0 +1,73 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_tbb_windef_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif /* __TBB_tbb_windef_H */
// Check that the target Windows version has all API calls requried for TBB.
// Do not increase the version in condition beyond 0x0500 without prior discussion!
#if defined(_WIN32_WINNT) && _WIN32_WINNT<0x0501
#error TBB is unable to run on old Windows versions; _WIN32_WINNT must be 0x0501 or greater.
#endif
#if !defined(_MT)
#error TBB requires linkage with multithreaded C/C++ runtime library. \
Choose multithreaded DLL runtime in project settings, or use /MD[d] compiler switch.
#endif
// Workaround for the problem with MVSC headers failing to define namespace std
namespace std {
using ::size_t; using ::ptrdiff_t;
}
#define __TBB_STRING_AUX(x) #x
#define __TBB_STRING(x) __TBB_STRING_AUX(x)
// Default setting of TBB_USE_DEBUG
#ifdef TBB_USE_DEBUG
# if TBB_USE_DEBUG
# if !defined(_DEBUG)
# pragma message(__FILE__ "(" __TBB_STRING(__LINE__) ") : Warning: Recommend using /MDd if compiling with TBB_USE_DEBUG!=0")
# endif
# else
# if defined(_DEBUG)
# pragma message(__FILE__ "(" __TBB_STRING(__LINE__) ") : Warning: Recommend using /MD if compiling with TBB_USE_DEBUG==0")
# endif
# endif
#endif
#if (__TBB_BUILD || __TBBMALLOC_BUILD) && !defined(__TBB_NO_IMPLICIT_LINKAGE)
#define __TBB_NO_IMPLICIT_LINKAGE 1
#endif
#if _MSC_VER
#if !__TBB_NO_IMPLICIT_LINKAGE
#ifdef __TBB_LIB_NAME
#pragma comment(lib, __TBB_STRING(__TBB_LIB_NAME))
#else
#ifdef _DEBUG
#pragma comment(lib, "tbb_debug.lib")
#else
#pragma comment(lib, "tbb.lib")
#endif
#endif
#endif
#endif

View File

@@ -0,0 +1,148 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__x86_eliding_mutex_impl_H
#define __TBB__x86_eliding_mutex_impl_H
#ifndef __TBB_spin_mutex_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#if ( __TBB_x86_32 || __TBB_x86_64 )
namespace tbb {
namespace interface7 {
namespace internal {
template<typename Mutex, bool is_rw>
class padded_mutex;
//! An eliding lock that occupies a single byte.
/** A x86_eliding_mutex is an HLE-enabled spin mutex. It is recommended to
put the mutex on a cache line that is not shared by the data it protects.
It should be used for locking short critical sections where the lock is
contended but the data it protects are not. If zero-initialized, the
mutex is considered unheld.
@ingroup synchronization */
class x86_eliding_mutex : tbb::internal::mutex_copy_deprecated_and_disabled {
//! 0 if lock is released, 1 if lock is acquired.
__TBB_atomic_flag flag;
friend class padded_mutex<x86_eliding_mutex, false>;
public:
//! Construct unacquired lock.
/** Equivalent to zero-initialization of *this. */
x86_eliding_mutex() : flag(0) {}
// bug in gcc 3.x.x causes syntax error in spite of the friend declaration above.
// Make the scoped_lock public in that case.
#if __TBB_USE_X86_ELIDING_MUTEX || __TBB_GCC_VERSION < 40000
#else
// by default we will not provide the scoped_lock interface. The user
// should use the padded version of the mutex. scoped_lock is used in
// padded_mutex template.
private:
#endif
// scoped_lock in padded_mutex<> is the interface to use.
//! Represents acquisition of a mutex.
class scoped_lock : tbb::internal::no_copy {
private:
//! Points to currently held mutex, or NULL if no lock is held.
x86_eliding_mutex* my_mutex;
public:
//! Construct without acquiring a mutex.
scoped_lock() : my_mutex(NULL) {}
//! Construct and acquire lock on a mutex.
scoped_lock( x86_eliding_mutex& m ) : my_mutex(NULL) { acquire(m); }
//! Acquire lock.
void acquire( x86_eliding_mutex& m ) {
__TBB_ASSERT( !my_mutex, "already holding a lock" );
my_mutex=&m;
my_mutex->lock();
}
//! Try acquiring lock (non-blocking)
/** Return true if lock acquired; false otherwise. */
bool try_acquire( x86_eliding_mutex& m ) {
__TBB_ASSERT( !my_mutex, "already holding a lock" );
bool result = m.try_lock();
if( result ) {
my_mutex = &m;
}
return result;
}
//! Release lock
void release() {
__TBB_ASSERT( my_mutex, "release on scoped_lock that is not holding a lock" );
my_mutex->unlock();
my_mutex = NULL;
}
//! Destroy lock. If holding a lock, releases the lock first.
~scoped_lock() {
if( my_mutex ) {
release();
}
}
};
#if __TBB_USE_X86_ELIDING_MUTEX || __TBB_GCC_VERSION < 40000
#else
public:
#endif /* __TBB_USE_X86_ELIDING_MUTEX */
// Mutex traits
static const bool is_rw_mutex = false;
static const bool is_recursive_mutex = false;
static const bool is_fair_mutex = false;
// ISO C++0x compatibility methods
//! Acquire lock
void lock() {
__TBB_LockByteElided(flag);
}
//! Try acquiring lock (non-blocking)
/** Return true if lock acquired; false otherwise. */
bool try_lock() {
return __TBB_TryLockByteElided(flag);
}
//! Release lock
void unlock() {
__TBB_UnlockByteElided( flag );
}
}; // end of x86_eliding_mutex
} // namespace internal
} // namespace interface7
} // namespace tbb
#endif /* ( __TBB_x86_32 || __TBB_x86_64 ) */
#endif /* __TBB__x86_eliding_mutex_impl_H */

View File

@@ -0,0 +1,225 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB__x86_rtm_rw_mutex_impl_H
#define __TBB__x86_rtm_rw_mutex_impl_H
#ifndef __TBB_spin_rw_mutex_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#if __TBB_TSX_AVAILABLE
#include "../tbb_stddef.h"
#include "../tbb_machine.h"
#include "../tbb_profiling.h"
#include "../spin_rw_mutex.h"
namespace tbb {
namespace interface8 {
namespace internal {
enum RTM_type {
RTM_not_in_mutex,
RTM_transacting_reader,
RTM_transacting_writer,
RTM_real_reader,
RTM_real_writer
};
static const unsigned long speculation_granularity = 64;
//! Fast, unfair, spinning speculation-enabled reader-writer lock with backoff and
// writer-preference
/** @ingroup synchronization */
class x86_rtm_rw_mutex: private spin_rw_mutex {
#if __TBB_USE_X86_RTM_RW_MUTEX || __TBB_GCC_VERSION < 40000
// bug in gcc 3.x.x causes syntax error in spite of the friend declaration below.
// Make the scoped_lock public in that case.
public:
#else
private:
#endif
friend class interface7::internal::padded_mutex<x86_rtm_rw_mutex,true>;
class scoped_lock; // should be private
friend class scoped_lock;
private:
//! @cond INTERNAL
//! Internal construct unacquired mutex.
void __TBB_EXPORTED_METHOD internal_construct();
//! Internal acquire write lock.
// only_speculate == true if we're doing a try_lock, else false.
void __TBB_EXPORTED_METHOD internal_acquire_writer(x86_rtm_rw_mutex::scoped_lock&, bool only_speculate=false);
//! Internal acquire read lock.
// only_speculate == true if we're doing a try_lock, else false.
void __TBB_EXPORTED_METHOD internal_acquire_reader(x86_rtm_rw_mutex::scoped_lock&, bool only_speculate=false);
//! Internal upgrade reader to become a writer.
bool __TBB_EXPORTED_METHOD internal_upgrade( x86_rtm_rw_mutex::scoped_lock& );
//! Out of line code for downgrading a writer to a reader.
bool __TBB_EXPORTED_METHOD internal_downgrade( x86_rtm_rw_mutex::scoped_lock& );
//! Internal try_acquire write lock.
bool __TBB_EXPORTED_METHOD internal_try_acquire_writer( x86_rtm_rw_mutex::scoped_lock& );
//! Internal release lock.
void __TBB_EXPORTED_METHOD internal_release( x86_rtm_rw_mutex::scoped_lock& );
static x86_rtm_rw_mutex* internal_get_mutex( const spin_rw_mutex::scoped_lock& lock )
{
return static_cast<x86_rtm_rw_mutex*>( lock.internal_get_mutex() );
}
static void internal_set_mutex( spin_rw_mutex::scoped_lock& lock, spin_rw_mutex* mtx )
{
lock.internal_set_mutex( mtx );
}
//! @endcond
public:
//! Construct unacquired mutex.
x86_rtm_rw_mutex() {
w_flag = false;
#if TBB_USE_THREADING_TOOLS
internal_construct();
#endif
}
#if TBB_USE_ASSERT
//! Empty destructor.
~x86_rtm_rw_mutex() {}
#endif /* TBB_USE_ASSERT */
// Mutex traits
static const bool is_rw_mutex = true;
static const bool is_recursive_mutex = false;
static const bool is_fair_mutex = false;
#if __TBB_USE_X86_RTM_RW_MUTEX || __TBB_GCC_VERSION < 40000
#else
// by default we will not provide the scoped_lock interface. The user
// should use the padded version of the mutex. scoped_lock is used in
// padded_mutex template.
private:
#endif
//! The scoped locking pattern
/** It helps to avoid the common problem of forgetting to release lock.
It also nicely provides the "node" for queuing locks. */
// Speculation-enabled scoped lock for spin_rw_mutex
// The idea is to be able to reuse the acquire/release methods of spin_rw_mutex
// and its scoped lock wherever possible. The only way to use a speculative lock is to use
// a scoped_lock. (because transaction_state must be local)
class scoped_lock : tbb::internal::no_copy {
friend class x86_rtm_rw_mutex;
spin_rw_mutex::scoped_lock my_scoped_lock;
RTM_type transaction_state;
public:
//! Construct lock that has not acquired a mutex.
/** Equivalent to zero-initialization of *this. */
scoped_lock() : my_scoped_lock(), transaction_state(RTM_not_in_mutex) {
}
//! Acquire lock on given mutex.
scoped_lock( x86_rtm_rw_mutex& m, bool write = true ) : my_scoped_lock(),
transaction_state(RTM_not_in_mutex) {
acquire(m, write);
}
//! Release lock (if lock is held).
~scoped_lock() {
if(transaction_state != RTM_not_in_mutex) release();
}
//! Acquire lock on given mutex.
void acquire( x86_rtm_rw_mutex& m, bool write = true ) {
if( write ) m.internal_acquire_writer(*this);
else m.internal_acquire_reader(*this);
}
//! Release lock
void release() {
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
__TBB_ASSERT( mutex, "lock is not acquired" );
__TBB_ASSERT( transaction_state!=RTM_not_in_mutex, "lock is not acquired" );
return mutex->internal_release(*this);
}
//! Upgrade reader to become a writer.
/** Returns whether the upgrade happened without releasing and re-acquiring the lock */
bool upgrade_to_writer() {
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
__TBB_ASSERT( mutex, "lock is not acquired" );
__TBB_ASSERT( transaction_state==RTM_transacting_reader || transaction_state==RTM_real_reader, "Invalid state for upgrade" );
return mutex->internal_upgrade(*this);
}
//! Downgrade writer to become a reader.
/** Returns whether the downgrade happened without releasing and re-acquiring the lock */
bool downgrade_to_reader() {
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
__TBB_ASSERT( mutex, "lock is not acquired" );
__TBB_ASSERT( transaction_state==RTM_transacting_writer || transaction_state==RTM_real_writer, "Invalid state for downgrade" );
return mutex->internal_downgrade(*this);
}
//! Attempt to acquire mutex.
/** returns true if successful. */
bool try_acquire( x86_rtm_rw_mutex& m, bool write = true ) {
#if TBB_USE_ASSERT
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
__TBB_ASSERT( !mutex, "lock is already acquired" );
#endif
// have to assign m to our mutex.
// cannot set the mutex, because try_acquire in spin_rw_mutex depends on it being NULL.
if(write) return m.internal_try_acquire_writer(*this);
// speculatively acquire the lock. If this fails, do try_acquire on the spin_rw_mutex.
m.internal_acquire_reader(*this, /*only_speculate=*/true);
if(transaction_state == RTM_transacting_reader) return true;
if( my_scoped_lock.try_acquire(m, false)) {
transaction_state = RTM_real_reader;
return true;
}
return false;
}
}; // class x86_rtm_rw_mutex::scoped_lock
// ISO C++0x compatibility methods not provided because we cannot maintain
// state about whether a thread is in a transaction.
private:
char pad[speculation_granularity-sizeof(spin_rw_mutex)]; // padding
// If true, writer holds the spin_rw_mutex.
tbb::atomic<bool> w_flag; // want this on a separate cache line
}; // x86_rtm_rw_mutex
} // namespace internal
} // namespace interface8
} // namespace tbb
#endif /* __TBB_TSX_AVAILABLE */
#endif /* __TBB__x86_rtm_rw_mutex_impl_H */

View File

@@ -0,0 +1,246 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_intrusive_list_H
#define _TBB_intrusive_list_H
#include "tbb/tbb_stddef.h"
namespace tbb {
namespace internal {
//! Data structure to be inherited by the types that can form intrusive lists.
/** Intrusive list is formed by means of the member_intrusive_list<T> template class.
Note that type T must derive from intrusive_list_node either publicly or
declare instantiation member_intrusive_list<T> as a friend.
This class implements a limited subset of std::list interface. **/
struct intrusive_list_node {
intrusive_list_node *my_prev_node,
*my_next_node;
#if TBB_USE_ASSERT
intrusive_list_node () { my_prev_node = my_next_node = this; }
#endif /* TBB_USE_ASSERT */
};
//! List of element of type T, where T is derived from intrusive_list_node
/** The class is not thread safe. **/
template <class List, class T>
class intrusive_list_base {
//! Pointer to the head node
intrusive_list_node my_head;
//! Number of list elements
size_t my_size;
static intrusive_list_node& node ( T& item ) { return List::node(item); }
static T& item ( intrusive_list_node* node ) { return List::item(node); }
template<class Iterator>
class iterator_impl {
Iterator& self () { return *static_cast<Iterator*>(this); }
//! Node the iterator points to at the moment
intrusive_list_node *my_pos;
protected:
iterator_impl (intrusive_list_node* pos )
: my_pos(pos)
{}
T& item () const {
return intrusive_list_base::item(my_pos);
}
public:
iterator_impl () : my_pos(NULL) {}
Iterator& operator = ( const Iterator& it ) {
return my_pos = it.my_pos;
}
Iterator& operator = ( const T& val ) {
return my_pos = &node(val);
}
bool operator == ( const Iterator& it ) const {
return my_pos == it.my_pos;
}
bool operator != ( const Iterator& it ) const {
return my_pos != it.my_pos;
}
Iterator& operator++ () {
my_pos = my_pos->my_next_node;
return self();
}
Iterator& operator-- () {
my_pos = my_pos->my_prev_node;
return self();
}
Iterator operator++ ( int ) {
Iterator result = self();
++(*this);
return result;
}
Iterator operator-- ( int ) {
Iterator result = self();
--(*this);
return result;
}
}; // intrusive_list_base::iterator_impl
void assert_ok () const {
__TBB_ASSERT( (my_head.my_prev_node == &my_head && !my_size) ||
(my_head.my_next_node != &my_head && my_size >0), "intrusive_list_base corrupted" );
#if TBB_USE_ASSERT >= 2
size_t i = 0;
for ( intrusive_list_node *n = my_head.my_next_node; n != &my_head; n = n->my_next_node )
++i;
__TBB_ASSERT( my_size == i, "Wrong size" );
#endif /* TBB_USE_ASSERT >= 2 */
}
public:
class iterator : public iterator_impl<iterator> {
template <class U, class V> friend class intrusive_list_base;
public:
iterator (intrusive_list_node* pos )
: iterator_impl<iterator>(pos )
{}
iterator () {}
T* operator-> () const { return &this->item(); }
T& operator* () const { return this->item(); }
}; // class iterator
class const_iterator : public iterator_impl<const_iterator> {
template <class U, class V> friend class intrusive_list_base;
public:
const_iterator (const intrusive_list_node* pos )
: iterator_impl<const_iterator>(const_cast<intrusive_list_node*>(pos) )
{}
const_iterator () {}
const T* operator-> () const { return &this->item(); }
const T& operator* () const { return this->item(); }
}; // class iterator
intrusive_list_base () : my_size(0) {
my_head.my_prev_node = &my_head;
my_head.my_next_node = &my_head;
}
bool empty () const { return my_head.my_next_node == &my_head; }
size_t size () const { return my_size; }
iterator begin () { return iterator(my_head.my_next_node); }
iterator end () { return iterator(&my_head); }
const_iterator begin () const { return const_iterator(my_head.my_next_node); }
const_iterator end () const { return const_iterator(&my_head); }
void push_front ( T& val ) {
__TBB_ASSERT( node(val).my_prev_node == &node(val) && node(val).my_next_node == &node(val),
"Object with intrusive list node can be part of only one intrusive list simultaneously" );
// An object can be part of only one intrusive list at the given moment via the given node member
node(val).my_prev_node = &my_head;
node(val).my_next_node = my_head.my_next_node;
my_head.my_next_node->my_prev_node = &node(val);
my_head.my_next_node = &node(val);
++my_size;
assert_ok();
}
void remove( T& val ) {
__TBB_ASSERT( node(val).my_prev_node != &node(val) && node(val).my_next_node != &node(val), "Element to remove is not in the list" );
__TBB_ASSERT( node(val).my_prev_node->my_next_node == &node(val) && node(val).my_next_node->my_prev_node == &node(val), "Element to remove is not in the list" );
--my_size;
node(val).my_next_node->my_prev_node = node(val).my_prev_node;
node(val).my_prev_node->my_next_node = node(val).my_next_node;
#if TBB_USE_ASSERT
node(val).my_prev_node = node(val).my_next_node = &node(val);
#endif
assert_ok();
}
iterator erase ( iterator it ) {
T& val = *it;
++it;
remove( val );
return it;
}
}; // intrusive_list_base
//! Double linked list of items of type T containing a member of type intrusive_list_node.
/** NodePtr is a member pointer to the node data field. Class U is either T or
a base class of T containing the node member. Default values exist for the sake
of a partial specialization working with inheritance case.
The list does not have ownership of its items. Its purpose is to avoid dynamic
memory allocation when forming lists of existing objects.
The class is not thread safe. **/
template <class T, class U, intrusive_list_node U::*NodePtr>
class memptr_intrusive_list : public intrusive_list_base<memptr_intrusive_list<T, U, NodePtr>, T>
{
friend class intrusive_list_base<memptr_intrusive_list<T, U, NodePtr>, T>;
static intrusive_list_node& node ( T& val ) { return val.*NodePtr; }
static T& item ( intrusive_list_node* node ) {
// Cannot use __TBB_offsetof (and consequently __TBB_get_object_ref) macro
// with *NodePtr argument because gcc refuses to interpret pasted "->" and "*"
// as member pointer dereferencing operator, and explicit usage of ## in
// __TBB_offsetof implementation breaks operations with normal member names.
return *reinterpret_cast<T*>((char*)node - ((ptrdiff_t)&(reinterpret_cast<T*>(0x1000)->*NodePtr) - 0x1000));
}
}; // intrusive_list<T, U, NodePtr>
//! Double linked list of items of type T that is derived from intrusive_list_node class.
/** The list does not have ownership of its items. Its purpose is to avoid dynamic
memory allocation when forming lists of existing objects.
The class is not thread safe. **/
template <class T>
class intrusive_list : public intrusive_list_base<intrusive_list<T>, T>
{
friend class intrusive_list_base<intrusive_list<T>, T>;
static intrusive_list_node& node ( T& val ) { return val; }
static T& item ( intrusive_list_node* node ) { return *static_cast<T*>(node); }
}; // intrusive_list<T>
} // namespace internal
} // namespace tbb
#endif /* _TBB_intrusive_list_H */

View File

@@ -0,0 +1,92 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if DO_ITT_NOTIFY
#if _WIN32||_WIN64
#ifndef UNICODE
#define UNICODE
#endif
#else
#pragma weak dlopen
#pragma weak dlsym
#pragma weak dlerror
#endif /* WIN */
#if __TBB_BUILD
extern "C" void ITT_DoOneTimeInitialization();
#define __itt_init_ittlib_name(x,y) (ITT_DoOneTimeInitialization(), true)
#elif __TBBMALLOC_BUILD
extern "C" void MallocInitializeITT();
#define __itt_init_ittlib_name(x,y) (MallocInitializeITT(), true)
#else
#error This file is expected to be used for either TBB or TBB allocator build.
#endif // __TBB_BUILD
#include "tools_api/ittnotify_static.c"
namespace tbb {
namespace internal {
int __TBB_load_ittnotify() {
return __itt_init_ittlib(NULL, // groups for:
(__itt_group_id)(__itt_group_sync // prepare/cancel/acquired/releasing
| __itt_group_thread // name threads
| __itt_group_stitch // stack stitching
#if __TBB_CPF_BUILD
| __itt_group_structure
#endif
));
}
}} // namespaces
#endif /* DO_ITT_NOTIFY */
#define __TBB_NO_IMPLICIT_LINKAGE 1
#include "itt_notify.h"
namespace tbb {
#if DO_ITT_NOTIFY
const tchar
*SyncType_GlobalLock = _T("TbbGlobalLock"),
*SyncType_Scheduler = _T("%Constant")
;
const tchar
*SyncObj_SchedulerInitialization = _T("TbbSchedulerInitialization"),
*SyncObj_SchedulersList = _T("TbbSchedulersList"),
*SyncObj_WorkerLifeCycleMgmt = _T("TBB Scheduler"),
*SyncObj_TaskStealingLoop = _T("TBB Scheduler"),
*SyncObj_WorkerTaskPool = _T("TBB Scheduler"),
*SyncObj_MasterTaskPool = _T("TBB Scheduler"),
*SyncObj_TaskPoolSpinning = _T("TBB Scheduler"),
*SyncObj_Mailbox = _T("TBB Scheduler"),
*SyncObj_TaskReturnList = _T("TBB Scheduler"),
*SyncObj_TaskStream = _T("TBB Scheduler"),
*SyncObj_ContextsList = _T("TBB Scheduler")
;
#endif /* DO_ITT_NOTIFY */
} // namespace tbb

View File

@@ -0,0 +1,128 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef _TBB_ITT_NOTIFY
#define _TBB_ITT_NOTIFY
#include "tbb/tbb_stddef.h"
#if DO_ITT_NOTIFY
#if _WIN32||_WIN64
#ifndef UNICODE
#define UNICODE
#endif
#endif /* WIN */
#ifndef INTEL_ITTNOTIFY_API_PRIVATE
#define INTEL_ITTNOTIFY_API_PRIVATE
#endif
#include "tools_api/ittnotify.h"
#include "tools_api/legacy/ittnotify.h"
extern "C" void __itt_fini_ittlib(void);
#if _WIN32||_WIN64
#undef _T
#undef __itt_event_create
#define __itt_event_create __itt_event_createA
#endif /* WIN */
#endif /* DO_ITT_NOTIFY */
#if !ITT_CALLER_NULL
#define ITT_CALLER_NULL ((__itt_caller)0)
#endif
namespace tbb {
//! Unicode support
#if (_WIN32||_WIN64) && !__MINGW32__
//! Unicode character type. Always wchar_t on Windows.
/** We do not use typedefs from Windows TCHAR family to keep consistence of TBB coding style. **/
typedef wchar_t tchar;
//! Standard Windows macro to markup the string literals.
#define _T(string_literal) L ## string_literal
#else /* !WIN */
typedef char tchar;
//! Standard Windows style macro to markup the string literals.
#define _T(string_literal) string_literal
#endif /* !WIN */
} // namespace tbb
#if DO_ITT_NOTIFY
namespace tbb {
//! Display names of internal synchronization types
extern const tchar
*SyncType_GlobalLock,
*SyncType_Scheduler;
//! Display names of internal synchronization components/scenarios
extern const tchar
*SyncObj_SchedulerInitialization,
*SyncObj_SchedulersList,
*SyncObj_WorkerLifeCycleMgmt,
*SyncObj_TaskStealingLoop,
*SyncObj_WorkerTaskPool,
*SyncObj_MasterTaskPool,
*SyncObj_TaskPoolSpinning,
*SyncObj_Mailbox,
*SyncObj_TaskReturnList,
*SyncObj_TaskStream,
*SyncObj_ContextsList
;
namespace internal {
void __TBB_EXPORTED_FUNC itt_set_sync_name_v3( void* obj, const tchar* name);
} // namespace internal
} // namespace tbb
// const_cast<void*>() is necessary to cast off volatility
#define ITT_NOTIFY(name,obj) __itt_notify_##name(const_cast<void*>(static_cast<volatile void*>(obj)))
#define ITT_THREAD_SET_NAME(name) __itt_thread_set_name(name)
#define ITT_FINI_ITTLIB() __itt_fini_ittlib()
#define ITT_SYNC_CREATE(obj, type, name) __itt_sync_create((void*)(obj), type, name, 2)
#define ITT_SYNC_RENAME(obj, name) __itt_sync_rename(obj, name)
#define ITT_STACK_CREATE(obj) obj = __itt_stack_caller_create()
#if __TBB_TASK_GROUP_CONTEXT
#define ITT_STACK(precond, name, obj) (precond) ? __itt_stack_##name(obj) : ((void)0);
#else
#define ITT_STACK(precond, name, obj) ((void)0)
#endif /* !__TBB_TASK_GROUP_CONTEXT */
#else /* !DO_ITT_NOTIFY */
#define ITT_NOTIFY(name,obj) ((void)0)
#define ITT_THREAD_SET_NAME(name) ((void)0)
#define ITT_FINI_ITTLIB() ((void)0)
#define ITT_SYNC_CREATE(obj, type, name) ((void)0)
#define ITT_SYNC_RENAME(obj, name) ((void)0)
#define ITT_STACK_CREATE(obj) ((void)0)
#define ITT_STACK(precond, name, obj) ((void)0)
#endif /* !DO_ITT_NOTIFY */
namespace tbb {
namespace internal {
int __TBB_load_ittnotify();
}}
#endif /* _TBB_ITT_NOTIFY */

View File

@@ -0,0 +1,49 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
{
global:
#define __TBB_SYMBOL( sym ) sym;
#include "lin32-tbb-export.lst"
local:
/* TBB symbols */
*3tbb*;
*__TBB*;
/* ITT symbols */
__itt_*;
/* Intel Compiler (libirc) symbols */
__intel_*;
_intel_*;
get_memcpy_largest_cachelinesize;
get_memcpy_largest_cache_size;
get_mem_ops_method;
init_mem_ops_method;
irc__get_msg;
irc__print;
override_mem_ops_method;
set_memcpy_largest_cachelinesize;
set_memcpy_largest_cache_size;
};

View File

@@ -0,0 +1,391 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
/* cache_aligned_allocator.cpp */
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEjjPv )
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Ej )
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
/* task.cpp v3 */
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEj )
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEj )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEj )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEj )
__TBB_SYMBOL( _ZTIN3tbb4taskE )
__TBB_SYMBOL( _ZTSN3tbb4taskE )
__TBB_SYMBOL( _ZTVN3tbb4taskE )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEij )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
#if __TBB_SCHEDULER_OBSERVER
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
#if __TBB_TASK_ARENA
/* arena.cpp */
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEi )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
#endif /* __TBB_TASK_ARENA */
#if !TBB_NO_LEGACY
/* task_v2.cpp */
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
#endif /* !TBB_NO_LEGACY */
/* Exception handling in task scheduler */
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEj )
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
#if __TBB_TASK_PRIORITY
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
#endif /* __TBB_TASK_PRIORITY */
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
#endif /* __TBB_TASK_GROUP_CONTEXT */
/* Symbols for exceptions thrown from TBB */
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
/* tbb_misc.cpp */
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
#if __TBB_x86_32
__TBB_SYMBOL( __TBB_machine_store8_slow_perf_warning )
__TBB_SYMBOL( __TBB_machine_store8_slow )
#endif
__TBB_SYMBOL( TBB_runtime_interface_version )
/* tbb_main.cpp */
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
#if __TBB_ITT_STRUCTURE_API
__TBB_SYMBOL( _ZN3tbb8internal22itt_make_task_group_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
__TBB_SYMBOL( _ZN3tbb8internal23itt_metadata_str_add_v7ENS0_15itt_domain_enumEPvyNS0_12string_indexEPKc )
__TBB_SYMBOL( _ZN3tbb8internal19itt_relation_add_v7ENS0_15itt_domain_enumEPvyNS0_12itt_relationES2_y )
__TBB_SYMBOL( _ZN3tbb8internal17itt_task_begin_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
__TBB_SYMBOL( _ZN3tbb8internal15itt_task_end_v7ENS0_15itt_domain_enumE )
#endif
/* pipeline.cpp */
__TBB_SYMBOL( _ZTIN3tbb6filterE )
__TBB_SYMBOL( _ZTSN3tbb6filterE )
__TBB_SYMBOL( _ZTVN3tbb6filterE )
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline3runEj )
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZN3tbb8pipeline3runEjRNS_18task_group_contextE )
#endif
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
/* queuing_rw_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
/* reader_writer_lock.cpp */
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
#if !TBB_NO_LEGACY
/* spin_rw_mutex.cpp v2 */
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
#endif
/* spin_rw_mutex v3 */
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
// x86_rtm_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
/* spin_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
/* mutex.cpp */
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
/* recursive_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
/* QueuingMutex.cpp */
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
/* critical_section.cpp */
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
#if !TBB_NO_LEGACY
/* concurrent_hash_map */
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
/* concurrent_queue.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityEij )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Ej )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
#endif
/* concurrent_queue v3 */
/* constructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Ej )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Ej )
/* destructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
/* typeinfo */
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
/* vtable */
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
/* methods */
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityEij )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
#if !TBB_NO_LEGACY
/* concurrent_vector.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_jPFvPvPKvjE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvjEb )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_jPFvPvjEPFvS4_PKvjESA_ )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEjjPFvPvjE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEjjj )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEjRj )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEjjPFvPvjE )
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
#endif
/* concurrent_vector v3 */
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_jPFvPvPKvjE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvjE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_jPFvPvjEPFvS4_PKvjESA_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEjjPFvPvPKvjES4_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEjjj )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEjRj )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEjjPFvPvPKvjES4_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEjPvPFvS2_jEPFvS2_PKvjE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEj )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEjjjPKvPFvPvjEPFvS4_S3_jE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEjjPFvPvPKvjES4_ )
/* tbb_thread */
#if __MINGW32__
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFjPvES2_ )
#else
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
#endif
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Ej )
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
#if __MINGW32__
/* condition_variable */
__TBB_SYMBOL( _ZN3tbb10interface58internal32internal_condition_variable_waitERNS1_14condvar_impl_tEPNS_5mutexEPKNS_10tick_count10interval_tE )
__TBB_SYMBOL( _ZN3tbb10interface58internal35internal_destroy_condition_variableERNS1_14condvar_impl_tE )
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_condition_variable_notify_allERNS1_14condvar_impl_tE )
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_condition_variable_notify_oneERNS1_14condvar_impl_tE )
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_initialize_condition_variableERNS1_14condvar_impl_tE )
#endif
#undef __TBB_SYMBOL

View File

@@ -0,0 +1,46 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
{
global:
#define __TBB_SYMBOL( sym ) sym;
#include "lin64-tbb-export.lst"
local:
/* TBB symbols */
*3tbb*;
*__TBB*;
/* ITT symbols */
__itt_*;
/* Intel Compiler (libirc) symbols */
__intel_*;
_intel_*;
get_msg_buf;
get_text_buf;
message_catalog;
print_buf;
irc__get_msg;
irc__print;
};

View File

@@ -0,0 +1,373 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
/* cache_aligned_allocator.cpp */
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
/* task.cpp v3 */
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
__TBB_SYMBOL( _ZTIN3tbb4taskE )
__TBB_SYMBOL( _ZTSN3tbb4taskE )
__TBB_SYMBOL( _ZTVN3tbb4taskE )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
#if __TBB_SCHEDULER_OBSERVER
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
#if __TBB_TASK_ARENA
/* arena.cpp */
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
#endif /* __TBB_TASK_ARENA */
#if !TBB_NO_LEGACY
/* task_v2.cpp */
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
#endif /* !TBB_NO_LEGACY */
/* Exception handling in task scheduler */
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
#if __TBB_TASK_PRIORITY
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
#endif /* __TBB_TASK_PRIORITY */
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
#endif /* __TBB_TASK_GROUP_CONTEXT */
/* Symbols for exceptions thrown from TBB */
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
/* tbb_misc.cpp */
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
__TBB_SYMBOL( TBB_runtime_interface_version )
/* tbb_main.cpp */
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
#if __TBB_ITT_STRUCTURE_API
__TBB_SYMBOL( _ZN3tbb8internal23itt_metadata_str_add_v7ENS0_15itt_domain_enumEPvyNS0_12string_indexEPKc )
__TBB_SYMBOL( _ZN3tbb8internal22itt_make_task_group_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
__TBB_SYMBOL( _ZN3tbb8internal17itt_task_begin_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
__TBB_SYMBOL( _ZN3tbb8internal19itt_relation_add_v7ENS0_15itt_domain_enumEPvyNS0_12itt_relationES2_y )
__TBB_SYMBOL( _ZN3tbb8internal15itt_task_end_v7ENS0_15itt_domain_enumE )
#endif
/* pipeline.cpp */
__TBB_SYMBOL( _ZTIN3tbb6filterE )
__TBB_SYMBOL( _ZTSN3tbb6filterE )
__TBB_SYMBOL( _ZTVN3tbb6filterE )
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
#endif
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
/* queuing_rw_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
/* reader_writer_lock.cpp */
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
#if !TBB_NO_LEGACY
/* spin_rw_mutex.cpp v2 */
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
#endif
// x86_rtm_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
/* spin_rw_mutex v3 */
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
/* spin_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
/* mutex.cpp */
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
/* recursive_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
/* QueuingMutex.cpp */
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
/* critical_section.cpp */
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
#if !TBB_NO_LEGACY
/* concurrent_hash_map */
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
/* concurrent_queue.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
#endif
/* concurrent_queue v3 */
/* constructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
/* destructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
/* typeinfo */
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
/* vtable */
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
/* methods */
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
#if !TBB_NO_LEGACY
/* concurrent_vector.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
#endif
/* concurrent_vector v3 */
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
/* tbb_thread */
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
#undef __TBB_SYMBOL

View File

@@ -0,0 +1,48 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
{
global:
#define __TBB_SYMBOL( sym ) sym;
#include "lin64ipf-tbb-export.lst"
local:
/* TBB symbols */
*3tbb*;
*__TBB*;
/* ITT symbols */
__itt_*;
/* Intel Compiler (libirc) symbols */
__intel_*;
_intel_*;
?0_memcopyA;
?0_memcopyDu;
?0_memcpyD;
?1__memcpy;
?1__memmove;
?1__serial_memmove;
memcpy;
memset;
};

View File

@@ -0,0 +1,408 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
/* cache_aligned_allocator.cpp */
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
/* task.cpp v3 */
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
__TBB_SYMBOL( _ZTIN3tbb4taskE )
__TBB_SYMBOL( _ZTSN3tbb4taskE )
__TBB_SYMBOL( _ZTVN3tbb4taskE )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
#if __TBB_SCHEDULER_OBSERVER
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
#if __TBB_TASK_ARENA
/* arena.cpp */
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
#endif /* __TBB_TASK_ARENA */
#if !TBB_NO_LEGACY
/* task_v2.cpp */
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
#endif /* !TBB_NO_LEGACY */
/* Exception handling in task scheduler */
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
#if __TBB_TASK_PRIORITY
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
#endif /* __TBB_TASK_PRIORITY */
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
#endif /* __TBB_TASK_GROUP_CONTEXT */
/* Symbols for exceptions thrown from TBB */
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
/* tbb_misc.cpp */
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
__TBB_SYMBOL( TBB_runtime_interface_version )
/* tbb_main.cpp */
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
/* pipeline.cpp */
__TBB_SYMBOL( _ZTIN3tbb6filterE )
__TBB_SYMBOL( _ZTSN3tbb6filterE )
__TBB_SYMBOL( _ZTVN3tbb6filterE )
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
#endif
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
/* queuing_rw_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
/* reader_writer_lock.cpp */
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
#if !TBB_NO_LEGACY
/* spin_rw_mutex.cpp v2 */
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
#endif
/* spin_rw_mutex v3 */
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
/* spin_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
/* mutex.cpp */
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
/* recursive_mutex.cpp */
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
/* QueuingMutex.cpp */
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
/* critical_section.cpp */
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
#if !TBB_NO_LEGACY
/* concurrent_hash_map */
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
/* concurrent_queue.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
#endif
/* concurrent_queue v3 */
/* constructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
/* destructors */
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
/* typeinfo */
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
/* vtable */
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
/* methods */
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
#if !TBB_NO_LEGACY
/* concurrent_vector.cpp v2 */
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
#endif
/* concurrent_vector v3 */
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
/* tbb_thread */
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
/* asm functions */
__TBB_SYMBOL( __TBB_machine_fetchadd1__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchadd2__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchadd4__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchadd8__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchstore1__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchstore2__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchstore4__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchstore8__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_fetchadd1acquire )
__TBB_SYMBOL( __TBB_machine_fetchadd1release )
__TBB_SYMBOL( __TBB_machine_fetchadd2acquire )
__TBB_SYMBOL( __TBB_machine_fetchadd2release )
__TBB_SYMBOL( __TBB_machine_fetchadd4acquire )
__TBB_SYMBOL( __TBB_machine_fetchadd4release )
__TBB_SYMBOL( __TBB_machine_fetchadd8acquire )
__TBB_SYMBOL( __TBB_machine_fetchadd8release )
__TBB_SYMBOL( __TBB_machine_fetchstore1acquire )
__TBB_SYMBOL( __TBB_machine_fetchstore1release )
__TBB_SYMBOL( __TBB_machine_fetchstore2acquire )
__TBB_SYMBOL( __TBB_machine_fetchstore2release )
__TBB_SYMBOL( __TBB_machine_fetchstore4acquire )
__TBB_SYMBOL( __TBB_machine_fetchstore4release )
__TBB_SYMBOL( __TBB_machine_fetchstore8acquire )
__TBB_SYMBOL( __TBB_machine_fetchstore8release )
__TBB_SYMBOL( __TBB_machine_cmpswp1acquire )
__TBB_SYMBOL( __TBB_machine_cmpswp1release )
__TBB_SYMBOL( __TBB_machine_cmpswp1__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_cmpswp2acquire )
__TBB_SYMBOL( __TBB_machine_cmpswp2release )
__TBB_SYMBOL( __TBB_machine_cmpswp2__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_cmpswp4acquire )
__TBB_SYMBOL( __TBB_machine_cmpswp4release )
__TBB_SYMBOL( __TBB_machine_cmpswp4__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_cmpswp8acquire )
__TBB_SYMBOL( __TBB_machine_cmpswp8release )
__TBB_SYMBOL( __TBB_machine_cmpswp8__TBB_full_fence )
__TBB_SYMBOL( __TBB_machine_lg )
__TBB_SYMBOL( __TBB_machine_lockbyte )
__TBB_SYMBOL( __TBB_machine_pause )
__TBB_SYMBOL( __TBB_machine_trylockbyte )
__TBB_SYMBOL( __TBB_machine_load8_relaxed )
__TBB_SYMBOL( __TBB_machine_store8_relaxed )
__TBB_SYMBOL( __TBB_machine_load4_relaxed )
__TBB_SYMBOL( __TBB_machine_store4_relaxed )
__TBB_SYMBOL( __TBB_machine_load2_relaxed )
__TBB_SYMBOL( __TBB_machine_store2_relaxed )
__TBB_SYMBOL( __TBB_machine_load1_relaxed )
__TBB_SYMBOL( __TBB_machine_store1_relaxed )
#undef __TBB_SYMBOL

View File

@@ -0,0 +1,23 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#define __TBB_SYMBOL( sym ) _##sym
#include "mac32-tbb-export.lst"

View File

@@ -0,0 +1,392 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
/*
Sometimes OS X* requires leading underscore (e. g. in export list file), but sometimes not
(e. g. when searching symbol in a dynamic library via dlsym()). Symbols in this file SHOULD
be listed WITHOUT one leading underscore. __TBB_SYMBOL macro should add underscore when
necessary, depending on the indended usage.
*/
// cache_aligned_allocator.cpp
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
// task.cpp v3
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
__TBB_SYMBOL( _ZTIN3tbb4taskE )
__TBB_SYMBOL( _ZTSN3tbb4taskE )
__TBB_SYMBOL( _ZTVN3tbb4taskE )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
#if __TBB_SCHEDULER_OBSERVER
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
#if __TBB_TASK_ARENA
/* arena.cpp */
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
#endif /* __TBB_TASK_ARENA */
#if !TBB_NO_LEGACY
// task_v2.cpp
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
#endif
// Exception handling in task scheduler
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
#if __TBB_TASK_PRIORITY
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
#endif /* __TBB_TASK_PRIORITY */
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
#endif /* __TBB_TASK_GROUP_CONTEXT */
// Symbols for exceptions thrown from TBB
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
__TBB_SYMBOL( _ZNSt13runtime_errorD1Ev )
__TBB_SYMBOL( _ZTISt13runtime_error )
__TBB_SYMBOL( _ZTSSt13runtime_error )
__TBB_SYMBOL( _ZNSt16invalid_argumentD1Ev )
__TBB_SYMBOL( _ZTISt16invalid_argument )
__TBB_SYMBOL( _ZTSSt16invalid_argument )
__TBB_SYMBOL( _ZNSt11range_errorD1Ev )
__TBB_SYMBOL( _ZTISt11range_error )
__TBB_SYMBOL( _ZTSSt11range_error )
__TBB_SYMBOL( _ZNSt12length_errorD1Ev )
__TBB_SYMBOL( _ZTISt12length_error )
__TBB_SYMBOL( _ZTSSt12length_error )
__TBB_SYMBOL( _ZNSt12out_of_rangeD1Ev )
__TBB_SYMBOL( _ZTISt12out_of_range )
__TBB_SYMBOL( _ZTSSt12out_of_range )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
// tbb_misc.cpp
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
#if __TBB_x86_32
__TBB_SYMBOL( __TBB_machine_store8_slow_perf_warning )
__TBB_SYMBOL( __TBB_machine_store8_slow )
#endif
__TBB_SYMBOL( TBB_runtime_interface_version )
// tbb_main.cpp
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
// pipeline.cpp
__TBB_SYMBOL( _ZTIN3tbb6filterE )
__TBB_SYMBOL( _ZTSN3tbb6filterE )
__TBB_SYMBOL( _ZTVN3tbb6filterE )
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
#endif
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
// queuing_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
// reader_writer_lock.cpp
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
#if !TBB_NO_LEGACY
// spin_rw_mutex.cpp v2
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
#endif
// spin_rw_mutex v3
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
// x86_rtm_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
// spin_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
// mutex.cpp
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
// recursive_mutex.cpp
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
// queuing_mutex.cpp
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
// critical_section.cpp
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
#if !TBB_NO_LEGACY
// concurrent_hash_map
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
// concurrent_queue.cpp v2
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityEim )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
#endif
// concurrent_queue v3
// constructors
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
// destructors
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
// typeinfo
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
// vtable
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
// methods
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityEim )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
#if !TBB_NO_LEGACY
// concurrent_vector.cpp v2
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
#endif
// concurrent_vector v3
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
// tbb_thread
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
#undef __TBB_SYMBOL

View File

@@ -0,0 +1,23 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#define __TBB_SYMBOL( sym ) _##sym
#include "mac64-tbb-export.lst"

View File

@@ -0,0 +1,389 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#include "tbb/tbb_config.h"
/*
Sometimes OS X* requires leading underscore (e. g. in export list file), but sometimes not
(e. g. when searching symbol in a dynamic library via dlsym()). Symbols in this file SHOULD
be listed WITHOUT one leading underscore. __TBB_SYMBOL macro should add underscore when
necessary, depending on the indended usage.
*/
// cache_aligned_allocator.cpp
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
// task.cpp v3
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
__TBB_SYMBOL( _ZTIN3tbb4taskE )
__TBB_SYMBOL( _ZTSN3tbb4taskE )
__TBB_SYMBOL( _ZTVN3tbb4taskE )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
#if __TBB_SCHEDULER_OBSERVER
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
#endif /* __TBB_SCHEDULER_OBSERVER */
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
#if __TBB_TASK_ARENA
/* arena.cpp */
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
#endif /* __TBB_TASK_ARENA */
#if !TBB_NO_LEGACY
// task_v2.cpp
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
#endif
// Exception handling in task scheduler
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
#if __TBB_TASK_PRIORITY
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
#endif /* __TBB_TASK_PRIORITY */
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
#endif /* __TBB_TASK_GROUP_CONTEXT */
// Symbols for exceptions thrown from TBB
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
__TBB_SYMBOL( _ZNSt13runtime_errorD1Ev )
__TBB_SYMBOL( _ZTISt13runtime_error )
__TBB_SYMBOL( _ZTSSt13runtime_error )
__TBB_SYMBOL( _ZNSt16invalid_argumentD1Ev )
__TBB_SYMBOL( _ZTISt16invalid_argument )
__TBB_SYMBOL( _ZTSSt16invalid_argument )
__TBB_SYMBOL( _ZNSt11range_errorD1Ev )
__TBB_SYMBOL( _ZTISt11range_error )
__TBB_SYMBOL( _ZTSSt11range_error )
__TBB_SYMBOL( _ZNSt12length_errorD1Ev )
__TBB_SYMBOL( _ZTISt12length_error )
__TBB_SYMBOL( _ZTSSt12length_error )
__TBB_SYMBOL( _ZNSt12out_of_rangeD1Ev )
__TBB_SYMBOL( _ZTISt12out_of_range )
__TBB_SYMBOL( _ZTSSt12out_of_range )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
// tbb_misc.cpp
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
__TBB_SYMBOL( TBB_runtime_interface_version )
// tbb_main.cpp
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
// pipeline.cpp
__TBB_SYMBOL( _ZTIN3tbb6filterE )
__TBB_SYMBOL( _ZTSN3tbb6filterE )
__TBB_SYMBOL( _ZTVN3tbb6filterE )
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
#if __TBB_TASK_GROUP_CONTEXT
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
#endif
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
// queuing_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
// reader_writer_lock.cpp
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
#if !TBB_NO_LEGACY
// spin_rw_mutex.cpp v2
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
#endif
// spin_rw_mutex v3
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
// x86_rtm_rw_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
// spin_mutex.cpp
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
// mutex.cpp
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
// recursive_mutex.cpp
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
// queuing_mutex.cpp
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
// critical_section.cpp
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
#if !TBB_NO_LEGACY
// concurrent_hash_map
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
// concurrent_queue.cpp v2
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
#endif
// concurrent_queue v3
// constructors
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
// destructors
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
// typeinfo
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
// vtable
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
// methods
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
#if !TBB_NO_LEGACY
// concurrent_vector.cpp v2
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
#endif
// concurrent_vector v3
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
// tbb_thread
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
#undef __TBB_SYMBOL

View File

@@ -0,0 +1,217 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
/*
Platform isolation layer for the ARMv7-a architecture.
*/
#ifndef __TBB_machine_H
#error Do not include this file directly; include tbb_machine.h instead
#endif
//TODO: is ARMv7 is the only version ever to support?
#if !(__ARM_ARCH_7A__)
#error compilation requires an ARMv7-a architecture.
#endif
#include <sys/param.h>
#include <unistd.h>
#define __TBB_WORDSIZE 4
// Traditionally ARM is little-endian.
// Note that, since only the layout of aligned 32-bit words is of interest,
// any apparent PDP-endianness of 32-bit words at half-word alignment or
// any little-endian ordering of big-endian 32-bit words in 64-bit quantities
// may be disregarded for this setting.
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#elif defined(__BYTE_ORDER__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
#else
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
#endif
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
#define __TBB_full_memory_fence() __asm__ __volatile__("dmb ish": : :"memory")
#define __TBB_control_consistency_helper() __TBB_full_memory_fence()
#define __TBB_acquire_consistency_helper() __TBB_full_memory_fence()
#define __TBB_release_consistency_helper() __TBB_full_memory_fence()
//--------------------------------------------------
// Compare and swap
//--------------------------------------------------
/**
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
* @param value value to assign *ptr to if *ptr==comparand
* @param comparand value to compare with *ptr
* @return value originally in memory at ptr, regardless of success
*/
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand )
{
int32_t oldval, res;
__TBB_full_memory_fence();
do {
__asm__ __volatile__(
"ldrex %1, [%3]\n"
"mov %0, #0\n"
"cmp %1, %4\n"
"it eq\n"
"strexeq %0, %5, [%3]\n"
: "=&r" (res), "=&r" (oldval), "+Qo" (*(volatile int32_t*)ptr)
: "r" ((int32_t *)ptr), "Ir" (comparand), "r" (value)
: "cc");
} while (res);
__TBB_full_memory_fence();
return oldval;
}
/**
* Atomic CAS for 64 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
* @param value value to assign *ptr to if *ptr==comparand
* @param comparand value to compare with *ptr
* @return value originally in memory at ptr, regardless of success
*/
static inline int64_t __TBB_machine_cmpswp8(volatile void *ptr, int64_t value, int64_t comparand )
{
int64_t oldval;
int32_t res;
__TBB_full_memory_fence();
do {
__asm__ __volatile__(
"mov %0, #0\n"
"ldrexd %1, %H1, [%3]\n"
"cmp %1, %4\n"
"it eq\n"
"cmpeq %H1, %H4\n"
"it eq\n"
"strexdeq %0, %5, %H5, [%3]"
: "=&r" (res), "=&r" (oldval), "+Qo" (*(volatile int64_t*)ptr)
: "r" ((int64_t *)ptr), "r" (comparand), "r" (value)
: "cc");
} while (res);
__TBB_full_memory_fence();
return oldval;
}
static inline int32_t __TBB_machine_fetchadd4(volatile void* ptr, int32_t addend)
{
unsigned long tmp;
int32_t result, tmp2;
__TBB_full_memory_fence();
__asm__ __volatile__(
"1: ldrex %0, [%4]\n"
" add %3, %0, %5\n"
" strex %1, %3, [%4]\n"
" cmp %1, #0\n"
" bne 1b\n"
: "=&r" (result), "=&r" (tmp), "+Qo" (*(volatile int32_t*)ptr), "=&r"(tmp2)
: "r" ((int32_t *)ptr), "Ir" (addend)
: "cc");
__TBB_full_memory_fence();
return result;
}
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend)
{
unsigned long tmp;
int64_t result, tmp2;
__TBB_full_memory_fence();
__asm__ __volatile__(
"1: ldrexd %0, %H0, [%4]\n"
" adds %3, %0, %5\n"
" adc %H3, %H0, %H5\n"
" strexd %1, %3, %H3, [%4]\n"
" cmp %1, #0\n"
" bne 1b"
: "=&r" (result), "=&r" (tmp), "+Qo" (*(volatile int64_t*)ptr), "=&r"(tmp2)
: "r" ((int64_t *)ptr), "r" (addend)
: "cc");
__TBB_full_memory_fence();
return result;
}
inline void __TBB_machine_pause (int32_t delay )
{
while(delay>0)
{
__TBB_compiler_fence();
delay--;
}
}
namespace tbb {
namespace internal {
template <typename T, size_t S>
struct machine_load_store_relaxed {
static inline T load ( const volatile T& location ) {
const T value = location;
/*
* An extra memory barrier is required for errata #761319
* Please see http://infocenter.arm.com/help/topic/com.arm.doc.uan0004a
*/
__TBB_acquire_consistency_helper();
return value;
}
static inline void store ( volatile T& location, T value ) {
location = value;
}
};
}} // namespaces internal, tbb
// Machine specific atomic operations
#define __TBB_CompareAndSwap4(P,V,C) __TBB_machine_cmpswp4(P,V,C)
#define __TBB_CompareAndSwap8(P,V,C) __TBB_machine_cmpswp8(P,V,C)
#define __TBB_Pause(V) __TBB_machine_pause(V)
// Use generics for some things
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_STORE 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1

View File

@@ -0,0 +1,131 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_generic_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_gcc_generic_H
#include <stdint.h>
#include <unistd.h>
#define __TBB_WORDSIZE __SIZEOF_POINTER__
#if __TBB_GCC_64BIT_ATOMIC_BUILTINS_BROKEN
#define __TBB_64BIT_ATOMICS 0
#endif
/** FPU control setting not available for non-Intel architectures on Android **/
#if __ANDROID__ && __TBB_generic_arch
#define __TBB_CPU_CTL_ENV_PRESENT 0
#endif
// __BYTE_ORDER__ is used in accordance with http://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html,
// but __BIG_ENDIAN__ or __LITTLE_ENDIAN__ may be more commonly found instead.
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#elif defined(__BYTE_ORDER__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
#else
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
#endif
/** As this generic implementation has absolutely no information about underlying
hardware, its performance most likely will be sub-optimal because of full memory
fence usages where a more lightweight synchronization means (or none at all)
could suffice. Thus if you use this header to enable TBB on a new platform,
consider forking it and relaxing below helpers as appropriate. **/
#define __TBB_acquire_consistency_helper() __sync_synchronize()
#define __TBB_release_consistency_helper() __sync_synchronize()
#define __TBB_full_memory_fence() __sync_synchronize()
#define __TBB_control_consistency_helper() __sync_synchronize()
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T) \
inline T __TBB_machine_cmpswp##S( volatile void *ptr, T value, T comparand ) { \
return __sync_val_compare_and_swap(reinterpret_cast<volatile T *>(ptr),comparand,value); \
} \
\
inline T __TBB_machine_fetchadd##S( volatile void *ptr, T value ) { \
return __sync_fetch_and_add(reinterpret_cast<volatile T *>(ptr),value); \
} \
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t)
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t)
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t)
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t)
#undef __TBB_MACHINE_DEFINE_ATOMICS
namespace tbb{ namespace internal { namespace gcc_builtins {
inline int clz(unsigned int x){ return __builtin_clz(x);}
inline int clz(unsigned long int x){ return __builtin_clzl(x);}
inline int clz(unsigned long long int x){ return __builtin_clzll(x);}
}}}
//gcc __builtin_clz builtin count _number_ of leading zeroes
static inline intptr_t __TBB_machine_lg( uintptr_t x ) {
return sizeof(x)*8 - tbb::internal::gcc_builtins::clz(x) -1 ;
}
static inline void __TBB_machine_or( volatile void *ptr, uintptr_t addend ) {
__sync_fetch_and_or(reinterpret_cast<volatile uintptr_t *>(ptr),addend);
}
static inline void __TBB_machine_and( volatile void *ptr, uintptr_t addend ) {
__sync_fetch_and_and(reinterpret_cast<volatile uintptr_t *>(ptr),addend);
}
typedef unsigned char __TBB_Flag;
typedef __TBB_atomic __TBB_Flag __TBB_atomic_flag;
inline bool __TBB_machine_try_lock_byte( __TBB_atomic_flag &flag ) {
return __sync_lock_test_and_set(&flag,1)==0;
}
inline void __TBB_machine_unlock_byte( __TBB_atomic_flag &flag ) {
__sync_lock_release(&flag);
}
// Machine specific atomic operations
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
#define __TBB_TryLockByte __TBB_machine_try_lock_byte
#define __TBB_UnlockByte __TBB_machine_unlock_byte
// Definition of other functions
#define __TBB_Log2(V) __TBB_machine_lg(V)
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#if __TBB_WORDSIZE==4
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
#endif
#if __TBB_x86_32 || __TBB_x86_64
#include "gcc_itsx.h"
#endif

View File

@@ -0,0 +1,100 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_machine_gcc_ia32_common_H
#define __TBB_machine_gcc_ia32_common_H
//TODO: Add a higher-level function, e.g. tbb::interal::log2(), into tbb_stddef.h, which
//uses __TBB_Log2 and contains the assert and remove the assert from here and all other
//platform-specific headers.
//TODO: Check if use of gcc intrinsic gives a better chance for cross call optimizations
template <typename T>
static inline intptr_t __TBB_machine_lg( T x ) {
__TBB_ASSERT(x>0, "The logarithm of a non-positive value is undefined.");
uintptr_t j;
__asm__("bsr %1,%0" : "=r"(j) : "r"((uintptr_t)x));
return j;
}
#define __TBB_Log2(V) __TBB_machine_lg(V)
#ifndef __TBB_Pause
//TODO: check if raising a ratio of pause instructions to loop control instructions
//(via e.g. loop unrolling) gives any benefit for HT. E.g, the current implementation
//does about 2 CPU-consuming instructions for every pause instruction. Perhaps for
//high pause counts it should use an unrolled loop to raise the ratio, and thus free
//up more integer cycles for the other hyperthread. On the other hand, if the loop is
//unrolled too far, it won't fit in the core's loop cache, and thus take away
//instruction decode slots from the other hyperthread.
//TODO: check if use of gcc __builtin_ia32_pause intrinsic gives a "some how" better performing code
static inline void __TBB_machine_pause( int32_t delay ) {
for (int32_t i = 0; i < delay; i++) {
__asm__ __volatile__("pause;");
}
return;
}
#define __TBB_Pause(V) __TBB_machine_pause(V)
#endif /* !__TBB_Pause */
// API to retrieve/update FPU control setting
#ifndef __TBB_CPU_CTL_ENV_PRESENT
#define __TBB_CPU_CTL_ENV_PRESENT 1
namespace tbb {
namespace internal {
class cpu_ctl_env {
private:
int mxcsr;
short x87cw;
static const int MXCSR_CONTROL_MASK = ~0x3f; /* all except last six status bits */
public:
bool operator!=( const cpu_ctl_env& ctl ) const { return mxcsr != ctl.mxcsr || x87cw != ctl.x87cw; }
void get_env() {
#if __TBB_ICC_12_0_INL_ASM_FSTCW_BROKEN
cpu_ctl_env loc_ctl;
__asm__ __volatile__ (
"stmxcsr %0\n\t"
"fstcw %1"
: "=m"(loc_ctl.mxcsr), "=m"(loc_ctl.x87cw)
);
*this = loc_ctl;
#else
__asm__ __volatile__ (
"stmxcsr %0\n\t"
"fstcw %1"
: "=m"(mxcsr), "=m"(x87cw)
);
#endif
mxcsr &= MXCSR_CONTROL_MASK;
}
void set_env() const {
__asm__ __volatile__ (
"ldmxcsr %0\n\t"
"fldcw %1"
: : "m"(mxcsr), "m"(x87cw)
);
}
};
} // namespace internal
} // namespace tbb
#endif /* !__TBB_CPU_CTL_ENV_PRESENT */
#include "gcc_itsx.h"
#endif /* __TBB_machine_gcc_ia32_common_H */

View File

@@ -0,0 +1,123 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_itsx_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_gcc_itsx_H
#define __TBB_OP_XACQUIRE 0xF2
#define __TBB_OP_XRELEASE 0xF3
#define __TBB_OP_LOCK 0xF0
#define __TBB_STRINGIZE_INTERNAL(arg) #arg
#define __TBB_STRINGIZE(arg) __TBB_STRINGIZE_INTERNAL(arg)
#ifdef __TBB_x86_64
#define __TBB_r_out "=r"
#else
#define __TBB_r_out "=q"
#endif
inline static uint8_t __TBB_machine_try_lock_elided( volatile uint8_t* lk )
{
uint8_t value = 1;
__asm__ volatile (".byte " __TBB_STRINGIZE(__TBB_OP_XACQUIRE)"; lock; xchgb %0, %1;"
: __TBB_r_out(value), "=m"(*lk) : "0"(value), "m"(*lk) : "memory" );
return uint8_t(value^1);
}
inline static void __TBB_machine_try_lock_elided_cancel()
{
// 'pause' instruction aborts HLE/RTM transactions
__asm__ volatile ("pause\n" : : : "memory" );
}
inline static void __TBB_machine_unlock_elided( volatile uint8_t* lk )
{
__asm__ volatile (".byte " __TBB_STRINGIZE(__TBB_OP_XRELEASE)"; movb $0, %0"
: "=m"(*lk) : "m"(*lk) : "memory" );
}
#if __TBB_TSX_INTRINSICS_PRESENT
#include <immintrin.h>
#define __TBB_machine_is_in_transaction _xtest
#define __TBB_machine_begin_transaction _xbegin
#define __TBB_machine_end_transaction _xend
#define __TBB_machine_transaction_conflict_abort() _xabort(0xff)
#else
/*!
* Check if the instruction is executed in a transaction or not
*/
inline static bool __TBB_machine_is_in_transaction()
{
int8_t res = 0;
#if __TBB_x86_32
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD6;\n"
"setz %0" : "=q"(res) : : "memory" );
#else
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD6;\n"
"setz %0" : "=r"(res) : : "memory" );
#endif
return res==0;
}
/*!
* Enter speculative execution mode.
* @return -1 on success
* abort cause ( or 0 ) on abort
*/
inline static uint32_t __TBB_machine_begin_transaction()
{
uint32_t res = ~uint32_t(0); // success value
__asm__ volatile ("1: .byte 0xC7; .byte 0xF8;\n" // XBEGIN <abort-offset>
" .long 2f-1b-6\n" // 2f-1b == difference in addresses of start
// of XBEGIN and the MOVL
// 2f - 1b - 6 == that difference minus the size of the
// XBEGIN instruction. This is the abort offset to
// 2: below.
" jmp 3f\n" // success (leave -1 in res)
"2: movl %%eax,%0\n" // store failure code in res
"3:"
:"=r"(res):"0"(res):"memory","%eax");
return res;
}
/*!
* Attempt to commit/end transaction
*/
inline static void __TBB_machine_end_transaction()
{
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD5" :::"memory"); // XEND
}
/*
* aborts with code 0xFF (lock already held)
*/
inline static void __TBB_machine_transaction_conflict_abort()
{
__asm__ volatile (".byte 0xC6; .byte 0xF8; .byte 0xFF" :::"memory");
}
#endif /* __TBB_TSX_INTRINSICS_PRESENT */

View File

@@ -0,0 +1,70 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
// TODO: revise by comparing with mac_ppc.h
#if !defined(__TBB_machine_H) || defined(__TBB_machine_ibm_aix51_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_ibm_aix51_H
#define __TBB_WORDSIZE 8
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG // assumption based on operating system
#include <stdint.h>
#include <unistd.h>
#include <sched.h>
extern "C" {
int32_t __TBB_machine_cas_32 (volatile void* ptr, int32_t value, int32_t comparand);
int64_t __TBB_machine_cas_64 (volatile void* ptr, int64_t value, int64_t comparand);
void __TBB_machine_flush ();
void __TBB_machine_lwsync ();
void __TBB_machine_isync ();
}
// Mapping of old entry point names retained for the sake of backward binary compatibility
#define __TBB_machine_cmpswp4 __TBB_machine_cas_32
#define __TBB_machine_cmpswp8 __TBB_machine_cas_64
#define __TBB_Yield() sched_yield()
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
#define __TBB_USE_GENERIC_FETCH_ADD 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#if __GNUC__
#define __TBB_control_consistency_helper() __asm__ __volatile__( "isync": : :"memory")
#define __TBB_acquire_consistency_helper() __asm__ __volatile__("lwsync": : :"memory")
#define __TBB_release_consistency_helper() __asm__ __volatile__("lwsync": : :"memory")
#define __TBB_full_memory_fence() __asm__ __volatile__( "sync": : :"memory")
#else
// IBM C++ Compiler does not support inline assembly
// TODO: Since XL 9.0 or earlier GCC syntax is supported. Replace with more
// lightweight implementation (like in mac_ppc.h)
#define __TBB_control_consistency_helper() __TBB_machine_isync ()
#define __TBB_acquire_consistency_helper() __TBB_machine_lwsync ()
#define __TBB_release_consistency_helper() __TBB_machine_lwsync ()
#define __TBB_full_memory_fence() __TBB_machine_flush ()
#endif

View File

@@ -0,0 +1,258 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_icc_generic_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#if ! __TBB_ICC_BUILTIN_ATOMICS_PRESENT
#error "Intel C++ Compiler of at least 12.0 version is needed to use ICC intrinsics port"
#endif
#define __TBB_machine_icc_generic_H
//ICC mimics the "native" target compiler
#if _MSC_VER
#include "msvc_ia32_common.h"
#else
#include "gcc_ia32_common.h"
#endif
//TODO: Make __TBB_WORDSIZE macro optional for ICC intrinsics port.
//As compiler intrinsics are used for all the operations it is possible to do.
#if __TBB_x86_32
#define __TBB_WORDSIZE 4
#else
#define __TBB_WORDSIZE 8
#endif
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
//__TBB_compiler_fence() defined just in case, as it seems not to be used on its own anywhere else
#if _MSC_VER
//TODO: any way to use same intrinsics on windows and linux?
#pragma intrinsic(_ReadWriteBarrier)
#define __TBB_compiler_fence() _ReadWriteBarrier()
#else
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
#endif
#ifndef __TBB_full_memory_fence
#if _MSC_VER
//TODO: any way to use same intrinsics on windows and linux?
#pragma intrinsic(_mm_mfence)
#define __TBB_full_memory_fence() _mm_mfence()
#else
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
#endif
#endif
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
namespace tbb { namespace internal {
//TODO: is there any way to reuse definition of memory_order enum from ICC instead of copy paste.
//however it seems unlikely that ICC will silently change exact enum values, as they are defined
//in the ISO exactly like this.
//TODO: add test that exact values of the enum are same as in the ISO C++11
typedef enum memory_order {
memory_order_relaxed, memory_order_consume, memory_order_acquire,
memory_order_release, memory_order_acq_rel, memory_order_seq_cst
} memory_order;
namespace icc_intrinsics_port {
template <typename T>
T convert_argument(T value){
return value;
}
//The overload below is needed to have explicit conversion of pointer to void* in argument list.
//compiler bug?
//TODO: add according broken macro and recheck with ICC 13.0 if the overload is still needed
template <typename T>
void* convert_argument(T* value){
return (void*)value;
}
}
//TODO: code below is a bit repetitive, consider simplifying it
template <typename T, size_t S>
struct machine_load_store {
static T load_with_acquire ( const volatile T& location ) {
return __atomic_load_explicit(&location, memory_order_acquire);
}
static void store_with_release ( volatile T &location, T value ) {
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_release);
}
};
template <typename T, size_t S>
struct machine_load_store_relaxed {
static inline T load ( const T& location ) {
return __atomic_load_explicit(&location, memory_order_relaxed);
}
static inline void store ( T& location, T value ) {
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_relaxed);
}
};
template <typename T, size_t S>
struct machine_load_store_seq_cst {
static T load ( const volatile T& location ) {
return __atomic_load_explicit(&location, memory_order_seq_cst);
}
static void store ( volatile T &location, T value ) {
__atomic_store_explicit(&location, value, memory_order_seq_cst);
}
};
}} // namespace tbb::internal
namespace tbb{ namespace internal { namespace icc_intrinsics_port{
typedef enum memory_order_map {
relaxed = memory_order_relaxed,
acquire = memory_order_acquire,
release = memory_order_release,
full_fence= memory_order_seq_cst
} memory_order_map;
}}}// namespace tbb::internal
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,M) \
inline T __TBB_machine_cmpswp##S##M( volatile void *ptr, T value, T comparand ) { \
__atomic_compare_exchange_strong_explicit( \
(T*)ptr \
,&comparand \
,value \
, tbb::internal::icc_intrinsics_port::M \
, tbb::internal::icc_intrinsics_port::M); \
return comparand; \
} \
\
inline T __TBB_machine_fetchstore##S##M(volatile void *ptr, T value) { \
return __atomic_exchange_explicit((T*)ptr, value, tbb::internal::icc_intrinsics_port::M); \
} \
\
inline T __TBB_machine_fetchadd##S##M(volatile void *ptr, T value) { \
return __atomic_fetch_add_explicit((T*)ptr, value, tbb::internal::icc_intrinsics_port::M); \
} \
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, full_fence)
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, acquire)
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, release)
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, relaxed)
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, full_fence)
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, acquire)
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, release)
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, relaxed)
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, full_fence)
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, acquire)
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, release)
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, relaxed)
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, full_fence)
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, acquire)
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, release)
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, relaxed)
#undef __TBB_MACHINE_DEFINE_ATOMICS
#define __TBB_USE_FENCED_ATOMICS 1
namespace tbb { namespace internal {
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(full_fence)
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(full_fence)
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(acquire)
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(release)
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(relaxed)
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(relaxed)
template <typename T>
struct machine_load_store<T,8> {
static T load_with_acquire ( const volatile T& location ) {
if( tbb::internal::is_aligned(&location,8)) {
return __atomic_load_explicit(&location, memory_order_acquire);
} else {
return __TBB_machine_generic_load8acquire(&location);
}
}
static void store_with_release ( volatile T &location, T value ) {
if( tbb::internal::is_aligned(&location,8)) {
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_release);
} else {
return __TBB_machine_generic_store8release(&location,value);
}
}
};
template <typename T>
struct machine_load_store_relaxed<T,8> {
static T load( const volatile T& location ) {
if( tbb::internal::is_aligned(&location,8)) {
return __atomic_load_explicit(&location, memory_order_relaxed);
} else {
return __TBB_machine_generic_load8relaxed(&location);
}
}
static void store( volatile T &location, T value ) {
if( tbb::internal::is_aligned(&location,8)) {
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_relaxed);
} else {
return __TBB_machine_generic_store8relaxed(&location,value);
}
}
};
template <typename T >
struct machine_load_store_seq_cst<T,8> {
static T load ( const volatile T& location ) {
if( tbb::internal::is_aligned(&location,8)) {
return __atomic_load_explicit(&location, memory_order_seq_cst);
} else {
return __TBB_machine_generic_load8full_fence(&location);
}
}
static void store ( volatile T &location, T value ) {
if( tbb::internal::is_aligned(&location,8)) {
__atomic_store_explicit(&location, value, memory_order_seq_cst);
} else {
return __TBB_machine_generic_store8full_fence(&location,value);
}
}
};
#endif
}} // namespace tbb::internal
template <typename T>
inline void __TBB_machine_OR( T *operand, T addend ) {
__atomic_fetch_or_explicit(operand, addend, tbb::internal::memory_order_seq_cst);
}
template <typename T>
inline void __TBB_machine_AND( T *operand, T addend ) {
__atomic_fetch_and_explicit(operand, addend, tbb::internal::memory_order_seq_cst);
}

View File

@@ -0,0 +1,84 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_machine_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#include <sched.h>
#define __TBB_Yield() sched_yield()
#include <unistd.h>
/* Futex definitions */
#include <sys/syscall.h>
#if defined(SYS_futex)
#define __TBB_USE_FUTEX 1
#include <limits.h>
#include <errno.h>
// Unfortunately, some versions of Linux do not have a header that defines FUTEX_WAIT and FUTEX_WAKE.
#ifdef FUTEX_WAIT
#define __TBB_FUTEX_WAIT FUTEX_WAIT
#else
#define __TBB_FUTEX_WAIT 0
#endif
#ifdef FUTEX_WAKE
#define __TBB_FUTEX_WAKE FUTEX_WAKE
#else
#define __TBB_FUTEX_WAKE 1
#endif
#ifndef __TBB_ASSERT
#error machine specific headers must be included after tbb_stddef.h
#endif
namespace tbb {
namespace internal {
inline int futex_wait( void *futex, int comparand ) {
int r = syscall( SYS_futex,futex,__TBB_FUTEX_WAIT,comparand,NULL,NULL,0 );
#if TBB_USE_ASSERT
int e = errno;
__TBB_ASSERT( r==0||r==EWOULDBLOCK||(r==-1&&(e==EAGAIN||e==EINTR)), "futex_wait failed." );
#endif /* TBB_USE_ASSERT */
return r;
}
inline int futex_wakeup_one( void *futex ) {
int r = ::syscall( SYS_futex,futex,__TBB_FUTEX_WAKE,1,NULL,NULL,0 );
__TBB_ASSERT( r==0||r==1, "futex_wakeup_one: more than one thread woken up?" );
return r;
}
inline int futex_wakeup_all( void *futex ) {
int r = ::syscall( SYS_futex,futex,__TBB_FUTEX_WAKE,INT_MAX,NULL,NULL,0 );
__TBB_ASSERT( r>=0, "futex_wakeup_all: error in waking up threads" );
return r;
}
} /* namespace internal */
} /* namespace tbb */
#endif /* SYS_futex */

View File

@@ -0,0 +1,232 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_ia32_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_linux_ia32_H
#include <stdint.h>
#include "gcc_ia32_common.h"
#define __TBB_WORDSIZE 4
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
#if __TBB_ICC_ASM_VOLATILE_BROKEN
#define __TBB_VOLATILE
#else
#define __TBB_VOLATILE volatile
#endif
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,X,R) \
static inline T __TBB_machine_cmpswp##S (volatile void *ptr, T value, T comparand ) \
{ \
T result; \
\
__asm__ __volatile__("lock\ncmpxchg" X " %2,%1" \
: "=a"(result), "=m"(*(__TBB_VOLATILE T*)ptr) \
: "q"(value), "0"(comparand), "m"(*(__TBB_VOLATILE T*)ptr) \
: "memory"); \
return result; \
} \
\
static inline T __TBB_machine_fetchadd##S(volatile void *ptr, T addend) \
{ \
T result; \
__asm__ __volatile__("lock\nxadd" X " %0,%1" \
: R (result), "=m"(*(__TBB_VOLATILE T*)ptr) \
: "0"(addend), "m"(*(__TBB_VOLATILE T*)ptr) \
: "memory"); \
return result; \
} \
\
static inline T __TBB_machine_fetchstore##S(volatile void *ptr, T value) \
{ \
T result; \
__asm__ __volatile__("lock\nxchg" X " %0,%1" \
: R (result), "=m"(*(__TBB_VOLATILE T*)ptr) \
: "0"(value), "m"(*(__TBB_VOLATILE T*)ptr) \
: "memory"); \
return result; \
} \
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t,"","=q")
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t,"","=r")
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t,"l","=r")
#if __INTEL_COMPILER
#pragma warning( push )
// reference to EBX in a function requiring stack alignment
#pragma warning( disable: 998 )
#endif
#if __TBB_GCC_CAS8_BUILTIN_INLINING_BROKEN
#define __TBB_IA32_CAS8_NOINLINE __attribute__ ((noinline))
#else
#define __TBB_IA32_CAS8_NOINLINE
#endif
static inline __TBB_IA32_CAS8_NOINLINE int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand ) {
//TODO: remove the extra part of condition once __TBB_GCC_BUILTIN_ATOMICS_PRESENT is lowered to gcc version 4.1.2
#if (__TBB_GCC_BUILTIN_ATOMICS_PRESENT || (__TBB_GCC_VERSION >= 40102)) && !__TBB_GCC_64BIT_ATOMIC_BUILTINS_BROKEN
return __sync_val_compare_and_swap( reinterpret_cast<volatile int64_t*>(ptr), comparand, value );
#else /* !__TBB_GCC_BUILTIN_ATOMICS_PRESENT */
//TODO: look like ICC 13.0 has some issues with this code, investigate it more deeply
int64_t result;
union {
int64_t i64;
int32_t i32[2];
};
i64 = value;
#if __PIC__
/* compiling position-independent code */
// EBX register preserved for compliance with position-independent code rules on IA32
int32_t tmp;
__asm__ __volatile__ (
"movl %%ebx,%2\n\t"
"movl %5,%%ebx\n\t"
#if __GNUC__==3
"lock\n\t cmpxchg8b %1\n\t"
#else
"lock\n\t cmpxchg8b (%3)\n\t"
#endif
"movl %2,%%ebx"
: "=A"(result)
, "=m"(*(__TBB_VOLATILE int64_t *)ptr)
, "=m"(tmp)
#if __GNUC__==3
: "m"(*(__TBB_VOLATILE int64_t *)ptr)
#else
: "SD"(ptr)
#endif
, "0"(comparand)
, "m"(i32[0]), "c"(i32[1])
: "memory"
#if __INTEL_COMPILER
,"ebx"
#endif
);
#else /* !__PIC__ */
__asm__ __volatile__ (
"lock\n\t cmpxchg8b %1\n\t"
: "=A"(result), "=m"(*(__TBB_VOLATILE int64_t *)ptr)
: "m"(*(__TBB_VOLATILE int64_t *)ptr)
, "0"(comparand)
, "b"(i32[0]), "c"(i32[1])
: "memory"
);
#endif /* __PIC__ */
return result;
#endif /* !__TBB_GCC_BUILTIN_ATOMICS_PRESENT */
}
#undef __TBB_IA32_CAS8_NOINLINE
#if __INTEL_COMPILER
#pragma warning( pop )
#endif // warning 998 is back
static inline void __TBB_machine_or( volatile void *ptr, uint32_t addend ) {
__asm__ __volatile__("lock\norl %1,%0" : "=m"(*(__TBB_VOLATILE uint32_t *)ptr) : "r"(addend), "m"(*(__TBB_VOLATILE uint32_t *)ptr) : "memory");
}
static inline void __TBB_machine_and( volatile void *ptr, uint32_t addend ) {
__asm__ __volatile__("lock\nandl %1,%0" : "=m"(*(__TBB_VOLATILE uint32_t *)ptr) : "r"(addend), "m"(*(__TBB_VOLATILE uint32_t *)ptr) : "memory");
}
//TODO: Check if it possible and profitable for IA-32 architecture on (Linux* and Windows*)
//to use of 64-bit load/store via floating point registers together with full fence
//for sequentially consistent load/store, instead of CAS.
#if __clang__
#define __TBB_fildq "fildll"
#define __TBB_fistpq "fistpll"
#else
#define __TBB_fildq "fildq"
#define __TBB_fistpq "fistpq"
#endif
static inline int64_t __TBB_machine_aligned_load8 (const volatile void *ptr) {
__TBB_ASSERT(tbb::internal::is_aligned(ptr,8),"__TBB_machine_aligned_load8 should be used with 8 byte aligned locations only \n");
int64_t result;
__asm__ __volatile__ ( __TBB_fildq " %1\n\t"
__TBB_fistpq " %0" : "=m"(result) : "m"(*(const __TBB_VOLATILE uint64_t*)ptr) : "memory" );
return result;
}
static inline void __TBB_machine_aligned_store8 (volatile void *ptr, int64_t value ) {
__TBB_ASSERT(tbb::internal::is_aligned(ptr,8),"__TBB_machine_aligned_store8 should be used with 8 byte aligned locations only \n");
// Aligned store
__asm__ __volatile__ ( __TBB_fildq " %1\n\t"
__TBB_fistpq " %0" : "=m"(*(__TBB_VOLATILE int64_t*)ptr) : "m"(value) : "memory" );
}
static inline int64_t __TBB_machine_load8 (const volatile void *ptr) {
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
if( tbb::internal::is_aligned(ptr,8)) {
#endif
return __TBB_machine_aligned_load8(ptr);
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
} else {
// Unaligned load
return __TBB_machine_cmpswp8(const_cast<void*>(ptr),0,0);
}
#endif
}
//! Handles misaligned 8-byte store
/** Defined in tbb_misc.cpp */
extern "C" void __TBB_machine_store8_slow( volatile void *ptr, int64_t value );
extern "C" void __TBB_machine_store8_slow_perf_warning( volatile void *ptr );
static inline void __TBB_machine_store8(volatile void *ptr, int64_t value) {
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
if( tbb::internal::is_aligned(ptr,8)) {
#endif
__TBB_machine_aligned_store8(ptr,value);
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
} else {
// Unaligned store
#if TBB_USE_PERFORMANCE_WARNINGS
__TBB_machine_store8_slow_perf_warning(ptr);
#endif /* TBB_USE_PERFORMANCE_WARNINGS */
__TBB_machine_store8_slow(ptr,value);
}
#endif
}
// Machine specific atomic operations
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
#define __TBB_USE_GENERIC_DWORD_FETCH_ADD 1
#define __TBB_USE_GENERIC_DWORD_FETCH_STORE 1
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1

View File

@@ -0,0 +1,181 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_ia64_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_linux_ia64_H
#include <stdint.h>
#include <ia64intrin.h>
#define __TBB_WORDSIZE 8
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#if __INTEL_COMPILER
#define __TBB_compiler_fence()
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper()
#define __TBB_release_consistency_helper()
#define __TBB_full_memory_fence() __mf()
#else
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
// Even though GCC imbues volatile loads with acquire semantics, it sometimes moves
// loads over the acquire fence. The following helpers stop such incorrect code motion.
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#define __TBB_full_memory_fence() __asm__ __volatile__("mf": : :"memory")
#endif /* !__INTEL_COMPILER */
// Most of the functions will be in a .s file
// TODO: revise dynamic_link, memory pools and etc. if the library dependency is removed.
extern "C" {
int8_t __TBB_machine_fetchadd1__TBB_full_fence (volatile void *ptr, int8_t addend);
int8_t __TBB_machine_fetchadd1acquire(volatile void *ptr, int8_t addend);
int8_t __TBB_machine_fetchadd1release(volatile void *ptr, int8_t addend);
int16_t __TBB_machine_fetchadd2__TBB_full_fence (volatile void *ptr, int16_t addend);
int16_t __TBB_machine_fetchadd2acquire(volatile void *ptr, int16_t addend);
int16_t __TBB_machine_fetchadd2release(volatile void *ptr, int16_t addend);
int32_t __TBB_machine_fetchadd4__TBB_full_fence (volatile void *ptr, int32_t value);
int32_t __TBB_machine_fetchadd4acquire(volatile void *ptr, int32_t addend);
int32_t __TBB_machine_fetchadd4release(volatile void *ptr, int32_t addend);
int64_t __TBB_machine_fetchadd8__TBB_full_fence (volatile void *ptr, int64_t value);
int64_t __TBB_machine_fetchadd8acquire(volatile void *ptr, int64_t addend);
int64_t __TBB_machine_fetchadd8release(volatile void *ptr, int64_t addend);
int8_t __TBB_machine_fetchstore1__TBB_full_fence (volatile void *ptr, int8_t value);
int8_t __TBB_machine_fetchstore1acquire(volatile void *ptr, int8_t value);
int8_t __TBB_machine_fetchstore1release(volatile void *ptr, int8_t value);
int16_t __TBB_machine_fetchstore2__TBB_full_fence (volatile void *ptr, int16_t value);
int16_t __TBB_machine_fetchstore2acquire(volatile void *ptr, int16_t value);
int16_t __TBB_machine_fetchstore2release(volatile void *ptr, int16_t value);
int32_t __TBB_machine_fetchstore4__TBB_full_fence (volatile void *ptr, int32_t value);
int32_t __TBB_machine_fetchstore4acquire(volatile void *ptr, int32_t value);
int32_t __TBB_machine_fetchstore4release(volatile void *ptr, int32_t value);
int64_t __TBB_machine_fetchstore8__TBB_full_fence (volatile void *ptr, int64_t value);
int64_t __TBB_machine_fetchstore8acquire(volatile void *ptr, int64_t value);
int64_t __TBB_machine_fetchstore8release(volatile void *ptr, int64_t value);
int8_t __TBB_machine_cmpswp1__TBB_full_fence (volatile void *ptr, int8_t value, int8_t comparand);
int8_t __TBB_machine_cmpswp1acquire(volatile void *ptr, int8_t value, int8_t comparand);
int8_t __TBB_machine_cmpswp1release(volatile void *ptr, int8_t value, int8_t comparand);
int16_t __TBB_machine_cmpswp2__TBB_full_fence (volatile void *ptr, int16_t value, int16_t comparand);
int16_t __TBB_machine_cmpswp2acquire(volatile void *ptr, int16_t value, int16_t comparand);
int16_t __TBB_machine_cmpswp2release(volatile void *ptr, int16_t value, int16_t comparand);
int32_t __TBB_machine_cmpswp4__TBB_full_fence (volatile void *ptr, int32_t value, int32_t comparand);
int32_t __TBB_machine_cmpswp4acquire(volatile void *ptr, int32_t value, int32_t comparand);
int32_t __TBB_machine_cmpswp4release(volatile void *ptr, int32_t value, int32_t comparand);
int64_t __TBB_machine_cmpswp8__TBB_full_fence (volatile void *ptr, int64_t value, int64_t comparand);
int64_t __TBB_machine_cmpswp8acquire(volatile void *ptr, int64_t value, int64_t comparand);
int64_t __TBB_machine_cmpswp8release(volatile void *ptr, int64_t value, int64_t comparand);
int64_t __TBB_machine_lg(uint64_t value);
void __TBB_machine_pause(int32_t delay);
bool __TBB_machine_trylockbyte( volatile unsigned char &ptr );
int64_t __TBB_machine_lockbyte( volatile unsigned char &ptr );
//! Retrieves the current RSE backing store pointer. IA64 specific.
void* __TBB_get_bsp();
int32_t __TBB_machine_load1_relaxed(const void *ptr);
int32_t __TBB_machine_load2_relaxed(const void *ptr);
int32_t __TBB_machine_load4_relaxed(const void *ptr);
int64_t __TBB_machine_load8_relaxed(const void *ptr);
void __TBB_machine_store1_relaxed(void *ptr, int32_t value);
void __TBB_machine_store2_relaxed(void *ptr, int32_t value);
void __TBB_machine_store4_relaxed(void *ptr, int32_t value);
void __TBB_machine_store8_relaxed(void *ptr, int64_t value);
} // extern "C"
// Mapping old entry points to the names corresponding to the new full_fence identifier.
#define __TBB_machine_fetchadd1full_fence __TBB_machine_fetchadd1__TBB_full_fence
#define __TBB_machine_fetchadd2full_fence __TBB_machine_fetchadd2__TBB_full_fence
#define __TBB_machine_fetchadd4full_fence __TBB_machine_fetchadd4__TBB_full_fence
#define __TBB_machine_fetchadd8full_fence __TBB_machine_fetchadd8__TBB_full_fence
#define __TBB_machine_fetchstore1full_fence __TBB_machine_fetchstore1__TBB_full_fence
#define __TBB_machine_fetchstore2full_fence __TBB_machine_fetchstore2__TBB_full_fence
#define __TBB_machine_fetchstore4full_fence __TBB_machine_fetchstore4__TBB_full_fence
#define __TBB_machine_fetchstore8full_fence __TBB_machine_fetchstore8__TBB_full_fence
#define __TBB_machine_cmpswp1full_fence __TBB_machine_cmpswp1__TBB_full_fence
#define __TBB_machine_cmpswp2full_fence __TBB_machine_cmpswp2__TBB_full_fence
#define __TBB_machine_cmpswp4full_fence __TBB_machine_cmpswp4__TBB_full_fence
#define __TBB_machine_cmpswp8full_fence __TBB_machine_cmpswp8__TBB_full_fence
// Mapping relaxed operations to the entry points implementing them.
/** On IA64 RMW operations implicitly have acquire semantics. Thus one cannot
actually have completely relaxed RMW operation here. **/
#define __TBB_machine_fetchadd1relaxed __TBB_machine_fetchadd1acquire
#define __TBB_machine_fetchadd2relaxed __TBB_machine_fetchadd2acquire
#define __TBB_machine_fetchadd4relaxed __TBB_machine_fetchadd4acquire
#define __TBB_machine_fetchadd8relaxed __TBB_machine_fetchadd8acquire
#define __TBB_machine_fetchstore1relaxed __TBB_machine_fetchstore1acquire
#define __TBB_machine_fetchstore2relaxed __TBB_machine_fetchstore2acquire
#define __TBB_machine_fetchstore4relaxed __TBB_machine_fetchstore4acquire
#define __TBB_machine_fetchstore8relaxed __TBB_machine_fetchstore8acquire
#define __TBB_machine_cmpswp1relaxed __TBB_machine_cmpswp1acquire
#define __TBB_machine_cmpswp2relaxed __TBB_machine_cmpswp2acquire
#define __TBB_machine_cmpswp4relaxed __TBB_machine_cmpswp4acquire
#define __TBB_machine_cmpswp8relaxed __TBB_machine_cmpswp8acquire
#define __TBB_MACHINE_DEFINE_ATOMICS(S,V) \
template <typename T> \
struct machine_load_store_relaxed<T,S> { \
static inline T load ( const T& location ) { \
return (T)__TBB_machine_load##S##_relaxed(&location); \
} \
static inline void store ( T& location, T value ) { \
__TBB_machine_store##S##_relaxed(&location, (V)value); \
} \
}
namespace tbb {
namespace internal {
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t);
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t);
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t);
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t);
}} // namespaces internal, tbb
#undef __TBB_MACHINE_DEFINE_ATOMICS
#define __TBB_USE_FENCED_ATOMICS 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
// Definition of Lock functions
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)
#define __TBB_LockByte(P) __TBB_machine_lockbyte(P)
// Definition of other utility functions
#define __TBB_Pause(V) __TBB_machine_pause(V)
#define __TBB_Log2(V) __TBB_machine_lg(V)

View File

@@ -0,0 +1,96 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_intel64_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_linux_intel64_H
#include <stdint.h>
#include "gcc_ia32_common.h"
#define __TBB_WORDSIZE 8
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#ifndef __TBB_full_memory_fence
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
#endif
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,X) \
static inline T __TBB_machine_cmpswp##S (volatile void *ptr, T value, T comparand ) \
{ \
T result; \
\
__asm__ __volatile__("lock\ncmpxchg" X " %2,%1" \
: "=a"(result), "=m"(*(volatile T*)ptr) \
: "q"(value), "0"(comparand), "m"(*(volatile T*)ptr) \
: "memory"); \
return result; \
} \
\
static inline T __TBB_machine_fetchadd##S(volatile void *ptr, T addend) \
{ \
T result; \
__asm__ __volatile__("lock\nxadd" X " %0,%1" \
: "=r"(result),"=m"(*(volatile T*)ptr) \
: "0"(addend), "m"(*(volatile T*)ptr) \
: "memory"); \
return result; \
} \
\
static inline T __TBB_machine_fetchstore##S(volatile void *ptr, T value) \
{ \
T result; \
__asm__ __volatile__("lock\nxchg" X " %0,%1" \
: "=r"(result),"=m"(*(volatile T*)ptr) \
: "0"(value), "m"(*(volatile T*)ptr) \
: "memory"); \
return result; \
} \
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t,"")
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t,"")
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t,"")
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t,"q")
#undef __TBB_MACHINE_DEFINE_ATOMICS
static inline void __TBB_machine_or( volatile void *ptr, uint64_t value ) {
__asm__ __volatile__("lock\norq %1,%0" : "=m"(*(volatile uint64_t*)ptr) : "r"(value), "m"(*(volatile uint64_t*)ptr) : "memory");
}
static inline void __TBB_machine_and( volatile void *ptr, uint64_t value ) {
__asm__ __volatile__("lock\nandq %1,%0" : "=m"(*(volatile uint64_t*)ptr) : "r"(value), "m"(*(volatile uint64_t*)ptr) : "memory");
}
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1

View File

@@ -0,0 +1,313 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_power_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_gcc_power_H
#include <stdint.h>
#include <unistd.h>
// TODO: rename to gcc_power.h?
// This file is for Power Architecture with compilers supporting GNU inline-assembler syntax (currently GNU g++ and IBM XL).
// Note that XL V9.0 (sometimes?) has trouble dealing with empty input and/or clobber lists, so they should be avoided.
#if __powerpc64__ || __ppc64__
// IBM XL documents __powerpc64__ (and __PPC64__).
// Apple documents __ppc64__ (with __ppc__ only on 32-bit).
#define __TBB_WORDSIZE 8
#else
#define __TBB_WORDSIZE 4
#endif
// Traditionally Power Architecture is big-endian.
// Little-endian could be just an address manipulation (compatibility with TBB not verified),
// or normal little-endian (on more recent systems). Embedded PowerPC systems may support
// page-specific endianness, but then one endianness must be hidden from TBB so that it still sees only one.
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#elif defined(__BYTE_ORDER__)
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
#else
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
#endif
// On Power Architecture, (lock-free) 64-bit atomics require 64-bit hardware:
#if __TBB_WORDSIZE==8
// Do not change the following definition, because TBB itself will use 64-bit atomics in 64-bit builds.
#define __TBB_64BIT_ATOMICS 1
#elif __bgp__
// Do not change the following definition, because this is known 32-bit hardware.
#define __TBB_64BIT_ATOMICS 0
#else
// To enable 64-bit atomics in 32-bit builds, set the value below to 1 instead of 0.
// You must make certain that the program will only use them on actual 64-bit hardware
// (which typically means that the entire program is only executed on such hardware),
// because their implementation involves machine instructions that are illegal elsewhere.
// The setting can be chosen independently per compilation unit,
// which also means that TBB itself does not need to be rebuilt.
// Alternatively (but only for the current architecture and TBB version),
// override the default as a predefined macro when invoking the compiler.
#ifndef __TBB_64BIT_ATOMICS
#define __TBB_64BIT_ATOMICS 0
#endif
#endif
inline int32_t __TBB_machine_cmpswp4 (volatile void *ptr, int32_t value, int32_t comparand )
{
int32_t result;
__asm__ __volatile__("sync\n"
"0:\n\t"
"lwarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
"cmpw %[res],%[cmp]\n\t" /* compare against comparand */
"bne- 1f\n\t" /* exit if not same */
"stwcx. %[val],0,%[ptr]\n\t" /* store new value */
"bne- 0b\n" /* retry if reservation lost */
"1:\n\t" /* the exit */
"isync"
: [res]"=&r"(result)
, "+m"(* (int32_t*) ptr) /* redundant with "memory" */
: [ptr]"r"(ptr)
, [val]"r"(value)
, [cmp]"r"(comparand)
: "memory" /* compiler full fence */
, "cr0" /* clobbered by cmp and/or stwcx. */
);
return result;
}
#if __TBB_WORDSIZE==8
inline int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand )
{
int64_t result;
__asm__ __volatile__("sync\n"
"0:\n\t"
"ldarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
"cmpd %[res],%[cmp]\n\t" /* compare against comparand */
"bne- 1f\n\t" /* exit if not same */
"stdcx. %[val],0,%[ptr]\n\t" /* store new value */
"bne- 0b\n" /* retry if reservation lost */
"1:\n\t" /* the exit */
"isync"
: [res]"=&r"(result)
, "+m"(* (int64_t*) ptr) /* redundant with "memory" */
: [ptr]"r"(ptr)
, [val]"r"(value)
, [cmp]"r"(comparand)
: "memory" /* compiler full fence */
, "cr0" /* clobbered by cmp and/or stdcx. */
);
return result;
}
#elif __TBB_64BIT_ATOMICS /* && __TBB_WORDSIZE==4 */
inline int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand )
{
int64_t result;
int64_t value_register, comparand_register, result_register; // dummy variables to allocate registers
__asm__ __volatile__("sync\n\t"
"ld %[val],%[valm]\n\t"
"ld %[cmp],%[cmpm]\n"
"0:\n\t"
"ldarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
"cmpd %[res],%[cmp]\n\t" /* compare against comparand */
"bne- 1f\n\t" /* exit if not same */
"stdcx. %[val],0,%[ptr]\n\t" /* store new value */
"bne- 0b\n" /* retry if reservation lost */
"1:\n\t" /* the exit */
"std %[res],%[resm]\n\t"
"isync"
: [resm]"=m"(result)
, [res] "=&r"( result_register)
, [val] "=&r"( value_register)
, [cmp] "=&r"(comparand_register)
, "+m"(* (int64_t*) ptr) /* redundant with "memory" */
: [ptr] "r"(ptr)
, [valm]"m"(value)
, [cmpm]"m"(comparand)
: "memory" /* compiler full fence */
, "cr0" /* clobbered by cmpd and/or stdcx. */
);
return result;
}
#endif /* __TBB_WORDSIZE==4 && __TBB_64BIT_ATOMICS */
#define __TBB_MACHINE_DEFINE_LOAD_STORE(S,ldx,stx,cmpx) \
template <typename T> \
struct machine_load_store<T,S> { \
static inline T load_with_acquire(const volatile T& location) { \
T result; \
__asm__ __volatile__(ldx " %[res],0(%[ptr])\n" \
"0:\n\t" \
cmpx " %[res],%[res]\n\t" \
"bne- 0b\n\t" \
"isync" \
: [res]"=r"(result) \
: [ptr]"b"(&location) /* cannot use register 0 here */ \
, "m"(location) /* redundant with "memory" */ \
: "memory" /* compiler acquire fence */ \
, "cr0" /* clobbered by cmpw/cmpd */); \
return result; \
} \
static inline void store_with_release(volatile T &location, T value) { \
__asm__ __volatile__("lwsync\n\t" \
stx " %[val],0(%[ptr])" \
: "=m"(location) /* redundant with "memory" */ \
: [ptr]"b"(&location) /* cannot use register 0 here */ \
, [val]"r"(value) \
: "memory"/*compiler release fence*/ /*(cr0 not affected)*/); \
} \
}; \
\
template <typename T> \
struct machine_load_store_relaxed<T,S> { \
static inline T load (const __TBB_atomic T& location) { \
T result; \
__asm__ __volatile__(ldx " %[res],0(%[ptr])" \
: [res]"=r"(result) \
: [ptr]"b"(&location) /* cannot use register 0 here */ \
, "m"(location) \
); /*(no compiler fence)*/ /*(cr0 not affected)*/ \
return result; \
} \
static inline void store (__TBB_atomic T &location, T value) { \
__asm__ __volatile__(stx " %[val],0(%[ptr])" \
: "=m"(location) \
: [ptr]"b"(&location) /* cannot use register 0 here */ \
, [val]"r"(value) \
); /*(no compiler fence)*/ /*(cr0 not affected)*/ \
} \
};
namespace tbb {
namespace internal {
__TBB_MACHINE_DEFINE_LOAD_STORE(1,"lbz","stb","cmpw")
__TBB_MACHINE_DEFINE_LOAD_STORE(2,"lhz","sth","cmpw")
__TBB_MACHINE_DEFINE_LOAD_STORE(4,"lwz","stw","cmpw")
#if __TBB_WORDSIZE==8
__TBB_MACHINE_DEFINE_LOAD_STORE(8,"ld" ,"std","cmpd")
#elif __TBB_64BIT_ATOMICS /* && __TBB_WORDSIZE==4 */
template <typename T>
struct machine_load_store<T,8> {
static inline T load_with_acquire(const volatile T& location) {
T result;
T result_register; // dummy variable to allocate a register
__asm__ __volatile__("ld %[res],0(%[ptr])\n\t"
"std %[res],%[resm]\n"
"0:\n\t"
"cmpd %[res],%[res]\n\t"
"bne- 0b\n\t"
"isync"
: [resm]"=m"(result)
, [res]"=&r"(result_register)
: [ptr]"b"(&location) /* cannot use register 0 here */
, "m"(location) /* redundant with "memory" */
: "memory" /* compiler acquire fence */
, "cr0" /* clobbered by cmpd */);
return result;
}
static inline void store_with_release(volatile T &location, T value) {
T value_register; // dummy variable to allocate a register
__asm__ __volatile__("lwsync\n\t"
"ld %[val],%[valm]\n\t"
"std %[val],0(%[ptr])"
: "=m"(location) /* redundant with "memory" */
, [val]"=&r"(value_register)
: [ptr]"b"(&location) /* cannot use register 0 here */
, [valm]"m"(value)
: "memory"/*compiler release fence*/ /*(cr0 not affected)*/);
}
};
struct machine_load_store_relaxed<T,8> {
static inline T load (const volatile T& location) {
T result;
T result_register; // dummy variable to allocate a register
__asm__ __volatile__("ld %[res],0(%[ptr])\n\t"
"std %[res],%[resm]"
: [resm]"=m"(result)
, [res]"=&r"(result_register)
: [ptr]"b"(&location) /* cannot use register 0 here */
, "m"(location)
); /*(no compiler fence)*/ /*(cr0 not affected)*/
return result;
}
static inline void store (volatile T &location, T value) {
T value_register; // dummy variable to allocate a register
__asm__ __volatile__("ld %[val],%[valm]\n\t"
"std %[val],0(%[ptr])"
: "=m"(location)
, [val]"=&r"(value_register)
: [ptr]"b"(&location) /* cannot use register 0 here */
, [valm]"m"(value)
); /*(no compiler fence)*/ /*(cr0 not affected)*/
}
};
#define __TBB_machine_load_store_relaxed_8
#endif /* __TBB_WORDSIZE==4 && __TBB_64BIT_ATOMICS */
}} // namespaces internal, tbb
#undef __TBB_MACHINE_DEFINE_LOAD_STORE
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
#define __TBB_USE_GENERIC_FETCH_ADD 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#define __TBB_control_consistency_helper() __asm__ __volatile__("isync": : :"memory")
#define __TBB_full_memory_fence() __asm__ __volatile__( "sync": : :"memory")
static inline intptr_t __TBB_machine_lg( uintptr_t x ) {
__TBB_ASSERT(x, "__TBB_Log2(0) undefined");
// cntlzd/cntlzw starts counting at 2^63/2^31 (ignoring any higher-order bits), and does not affect cr0
#if __TBB_WORDSIZE==8
__asm__ __volatile__ ("cntlzd %0,%0" : "+r"(x));
return 63-static_cast<intptr_t>(x);
#else
__asm__ __volatile__ ("cntlzw %0,%0" : "+r"(x));
return 31-static_cast<intptr_t>(x);
#endif
}
#define __TBB_Log2(V) __TBB_machine_lg(V)
// Assumes implicit alignment for any 32-bit value
typedef uint32_t __TBB_Flag;
#define __TBB_Flag __TBB_Flag
inline bool __TBB_machine_trylockbyte( __TBB_atomic __TBB_Flag &flag ) {
return __TBB_machine_cmpswp4(&flag,1,0)==0;
}
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)

View File

@@ -0,0 +1,133 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_macos_common_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_macos_common_H
#include <sched.h>
#define __TBB_Yield() sched_yield()
// __TBB_HardwareConcurrency
#include <sys/types.h>
#include <sys/sysctl.h>
static inline int __TBB_macos_available_cpu() {
int name[2] = {CTL_HW, HW_AVAILCPU};
int ncpu;
size_t size = sizeof(ncpu);
sysctl( name, 2, &ncpu, &size, NULL, 0 );
return ncpu;
}
#define __TBB_HardwareConcurrency() __TBB_macos_available_cpu()
#ifndef __TBB_full_memory_fence
// TBB has not recognized the architecture (none of the architecture abstraction
// headers was included).
#define __TBB_UnknownArchitecture 1
#endif
#if __TBB_UnknownArchitecture
// Implementation of atomic operations based on OS provided primitives
#include <libkern/OSAtomic.h>
static inline int64_t __TBB_machine_cmpswp8_OsX(volatile void *ptr, int64_t value, int64_t comparand)
{
__TBB_ASSERT( tbb::internal::is_aligned(ptr,8), "address not properly aligned for OS X* atomics");
int64_t* address = (int64_t*)ptr;
while( !OSAtomicCompareAndSwap64Barrier(comparand, value, address) ){
#if __TBB_WORDSIZE==8
int64_t snapshot = *address;
#else
int64_t snapshot = OSAtomicAdd64( 0, address );
#endif
if( snapshot!=comparand ) return snapshot;
}
return comparand;
}
#define __TBB_machine_cmpswp8 __TBB_machine_cmpswp8_OsX
#endif /* __TBB_UnknownArchitecture */
#if __TBB_UnknownArchitecture
#ifndef __TBB_WORDSIZE
#define __TBB_WORDSIZE 4
#endif
#ifdef __TBB_ENDIANNESS
// Already determined based on hardware architecture.
#elif __BIG_ENDIAN__
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
#elif __LITTLE_ENDIAN__
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#else
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
#endif
/** As this generic implementation has absolutely no information about underlying
hardware, its performance most likely will be sub-optimal because of full memory
fence usages where a more lightweight synchronization means (or none at all)
could suffice. Thus if you use this header to enable TBB on a new platform,
consider forking it and relaxing below helpers as appropriate. **/
#define __TBB_control_consistency_helper() OSMemoryBarrier()
#define __TBB_acquire_consistency_helper() OSMemoryBarrier()
#define __TBB_release_consistency_helper() OSMemoryBarrier()
#define __TBB_full_memory_fence() OSMemoryBarrier()
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand)
{
__TBB_ASSERT( tbb::internal::is_aligned(ptr,4), "address not properly aligned for OS X* atomics");
int32_t* address = (int32_t*)ptr;
while( !OSAtomicCompareAndSwap32Barrier(comparand, value, address) ){
int32_t snapshot = *address;
if( snapshot!=comparand ) return snapshot;
}
return comparand;
}
static inline int32_t __TBB_machine_fetchadd4(volatile void *ptr, int32_t addend)
{
__TBB_ASSERT( tbb::internal::is_aligned(ptr,4), "address not properly aligned for OS X* atomics");
return OSAtomicAdd32Barrier(addend, (int32_t*)ptr) - addend;
}
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend)
{
__TBB_ASSERT( tbb::internal::is_aligned(ptr,8), "address not properly aligned for OS X* atomics");
return OSAtomicAdd64Barrier(addend, (int64_t*)ptr) - addend;
}
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#if __TBB_WORDSIZE == 4
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
#endif
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#endif /* __TBB_UnknownArchitecture */

View File

@@ -0,0 +1,61 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_mic_common_H
#define __TBB_mic_common_H
#ifndef __TBB_machine_H
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#if ! __TBB_DEFINE_MIC
#error mic_common.h should be included only when building for Intel(R) Many Integrated Core Architecture
#endif
#ifndef __TBB_PREFETCHING
#define __TBB_PREFETCHING 1
#endif
#if __TBB_PREFETCHING
#include <immintrin.h>
#define __TBB_cl_prefetch(p) _mm_prefetch((const char*)p, _MM_HINT_T1)
#define __TBB_cl_evict(p) _mm_clevict(p, _MM_HINT_T1)
#endif
/** Intel(R) Many Integrated Core Architecture does not support mfence and pause instructions **/
#define __TBB_full_memory_fence() __asm__ __volatile__("lock; addl $0,(%%rsp)":::"memory")
#define __TBB_Pause(x) _mm_delay_32(16*(x))
#define __TBB_STEALING_PAUSE 1500/16
#include <sched.h>
#define __TBB_Yield() sched_yield()
// low-level timing intrinsic and its type
#define __TBB_machine_time_stamp() _rdtsc()
typedef uint64_t machine_tsc_t;
/** Specifics **/
#define __TBB_STEALING_ABORT_ON_CONTENTION 1
#define __TBB_YIELD2P 1
#define __TBB_HOARD_NONLOCAL_TASKS 1
#if ! ( __FreeBSD__ || __linux__ )
#error Intel(R) Many Integrated Core Compiler does not define __FreeBSD__ or __linux__ anymore. Check for the __TBB_XXX_BROKEN defined under __FreeBSD__ or __linux__.
#endif /* ! ( __FreeBSD__ || __linux__ ) */
#endif /* __TBB_mic_common_H */

View File

@@ -0,0 +1,171 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_msvc_armv7_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_msvc_armv7_H
#include <intrin.h>
#include <float.h>
#define __TBB_WORDSIZE 4
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
#if defined(TBB_WIN32_USE_CL_BUILTINS)
// We can test this on _M_IX86
#pragma intrinsic(_ReadWriteBarrier)
#pragma intrinsic(_mm_mfence)
#define __TBB_compiler_fence() _ReadWriteBarrier()
#define __TBB_full_memory_fence() _mm_mfence()
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#else
//Now __dmb(_ARM_BARRIER_SY) is used for both compiler and memory fences
//This might be changed later after testing
#define __TBB_compiler_fence() __dmb(_ARM_BARRIER_SY)
#define __TBB_full_memory_fence() __dmb(_ARM_BARRIER_SY)
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_full_memory_fence()
#define __TBB_release_consistency_helper() __TBB_full_memory_fence()
#endif
//--------------------------------------------------
// Compare and swap
//--------------------------------------------------
/**
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
* @param value value to assign *ptr to if *ptr==comparand
* @param comparand value to compare with *ptr
* @return value originally in memory at ptr, regardless of success
*/
#define __TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(S,T,F) \
inline T __TBB_machine_cmpswp##S( volatile void *ptr, T value, T comparand ) { \
return _InterlockedCompareExchange##F(reinterpret_cast<volatile T *>(ptr),value,comparand); \
} \
#define __TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(S,T,F) \
inline T __TBB_machine_fetchadd##S( volatile void *ptr, T value ) { \
return _InterlockedExchangeAdd##F(reinterpret_cast<volatile T *>(ptr),value); \
} \
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(1,char,8)
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(2,short,16)
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(4,long,)
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(8,__int64,64)
__TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(4,long,)
#if defined(TBB_WIN32_USE_CL_BUILTINS)
// No _InterlockedExchangeAdd64 intrinsic on _M_IX86
#define __TBB_64BIT_ATOMICS 0
#else
__TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(8,__int64,64)
#endif
inline void __TBB_machine_pause (int32_t delay )
{
while(delay>0)
{
__TBB_compiler_fence();
delay--;
}
}
// API to retrieve/update FPU control setting
#define __TBB_CPU_CTL_ENV_PRESENT 1
namespace tbb {
namespace internal {
template <typename T, size_t S>
struct machine_load_store_relaxed {
static inline T load ( const volatile T& location ) {
const T value = location;
/*
* An extra memory barrier is required for errata #761319
* Please see http://infocenter.arm.com/help/topic/com.arm.doc.uan0004a
*/
__TBB_acquire_consistency_helper();
return value;
}
static inline void store ( volatile T& location, T value ) {
location = value;
}
};
class cpu_ctl_env {
private:
unsigned int my_ctl;
public:
bool operator!=( const cpu_ctl_env& ctl ) const { return my_ctl != ctl.my_ctl; }
void get_env() { my_ctl = _control87(0, 0); }
void set_env() const { _control87( my_ctl, ~0U ); }
};
} // namespace internal
} // namespaces tbb
// Machine specific atomic operations
#define __TBB_CompareAndSwap4(P,V,C) __TBB_machine_cmpswp4(P,V,C)
#define __TBB_CompareAndSwap8(P,V,C) __TBB_machine_cmpswp8(P,V,C)
#define __TBB_Pause(V) __TBB_machine_pause(V)
// Use generics for some things
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_STORE 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#if defined(TBB_WIN32_USE_CL_BUILTINS)
#if !__TBB_WIN8UI_SUPPORT
extern "C" __declspec(dllimport) int __stdcall SwitchToThread( void );
#define __TBB_Yield() SwitchToThread()
#else
#include<thread>
#define __TBB_Yield() std::this_thread::yield()
#endif
#else
#define __TBB_Yield() __yield()
#endif
// Machine specific atomic operations
#define __TBB_AtomicOR(P,V) __TBB_machine_OR(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_AND(P,V)
template <typename T1,typename T2>
inline void __TBB_machine_OR( T1 *operand, T2 addend ) {
_InterlockedOr((long volatile *)operand, (long)addend);
}
template <typename T1,typename T2>
inline void __TBB_machine_AND( T1 *operand, T2 addend ) {
_InterlockedAnd((long volatile *)operand, (long)addend);
}

View File

@@ -0,0 +1,216 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_machine_msvc_ia32_common_H
#define __TBB_machine_msvc_ia32_common_H
#include <intrin.h>
//TODO: consider moving this macro to tbb_config.h and used there MSVC asm is used
#if !_M_X64 || __INTEL_COMPILER
#define __TBB_X86_MSVC_INLINE_ASM_AVAILABLE 1
#if _M_X64
#define __TBB_r(reg_name) r##reg_name
#else
#define __TBB_r(reg_name) e##reg_name
#endif
#else
//MSVC in x64 mode does not accept inline assembler
#define __TBB_X86_MSVC_INLINE_ASM_AVAILABLE 0
#endif
#define __TBB_NO_X86_MSVC_INLINE_ASM_MSG "The compiler being used is not supported (outdated?)"
#if (_MSC_VER >= 1300) || (__INTEL_COMPILER) //Use compiler intrinsic when available
#define __TBB_PAUSE_USE_INTRINSIC 1
#pragma intrinsic(_mm_pause)
namespace tbb { namespace internal { namespace intrinsics { namespace msvc {
static inline void __TBB_machine_pause (uintptr_t delay ) {
for (;delay>0; --delay )
_mm_pause();
}
}}}}
#else
#if !__TBB_X86_MSVC_INLINE_ASM_AVAILABLE
#error __TBB_NO_X86_MSVC_INLINE_ASM_MSG
#endif
namespace tbb { namespace internal { namespace inline_asm { namespace msvc {
static inline void __TBB_machine_pause (uintptr_t delay ) {
_asm
{
mov __TBB_r(ax), delay
__TBB_L1:
pause
add __TBB_r(ax), -1
jne __TBB_L1
}
return;
}
}}}}
#endif
static inline void __TBB_machine_pause (uintptr_t delay ){
#if __TBB_PAUSE_USE_INTRINSIC
tbb::internal::intrinsics::msvc::__TBB_machine_pause(delay);
#else
tbb::internal::inline_asm::msvc::__TBB_machine_pause(delay);
#endif
}
//TODO: move this function to windows_api.h or to place where it is used
#if (_MSC_VER<1400) && (!_WIN64) && (__TBB_X86_MSVC_INLINE_ASM_AVAILABLE)
static inline void* __TBB_machine_get_current_teb () {
void* pteb;
__asm mov eax, fs:[0x18]
__asm mov pteb, eax
return pteb;
}
#endif
#if ( _MSC_VER>=1400 && !defined(__INTEL_COMPILER) ) || (__INTEL_COMPILER>=1200)
// MSVC did not have this intrinsic prior to VC8.
// ICL 11.1 fails to compile a TBB example if __TBB_Log2 uses the intrinsic.
#define __TBB_LOG2_USE_BSR_INTRINSIC 1
#if _M_X64
#define __TBB_BSR_INTRINSIC _BitScanReverse64
#else
#define __TBB_BSR_INTRINSIC _BitScanReverse
#endif
#pragma intrinsic(__TBB_BSR_INTRINSIC)
namespace tbb { namespace internal { namespace intrinsics { namespace msvc {
inline uintptr_t __TBB_machine_lg( uintptr_t i ){
unsigned long j;
__TBB_BSR_INTRINSIC( &j, i );
return j;
}
}}}}
#else
#if !__TBB_X86_MSVC_INLINE_ASM_AVAILABLE
#error __TBB_NO_X86_MSVC_INLINE_ASM_MSG
#endif
namespace tbb { namespace internal { namespace inline_asm { namespace msvc {
inline uintptr_t __TBB_machine_lg( uintptr_t i ){
uintptr_t j;
__asm
{
bsr __TBB_r(ax), i
mov j, __TBB_r(ax)
}
return j;
}
}}}}
#endif
static inline intptr_t __TBB_machine_lg( uintptr_t i ) {
#if __TBB_LOG2_USE_BSR_INTRINSIC
return tbb::internal::intrinsics::msvc::__TBB_machine_lg(i);
#else
return tbb::internal::inline_asm::msvc::__TBB_machine_lg(i);
#endif
}
// API to retrieve/update FPU control setting
#define __TBB_CPU_CTL_ENV_PRESENT 1
namespace tbb { namespace internal { class cpu_ctl_env; } }
#if __TBB_X86_MSVC_INLINE_ASM_AVAILABLE
inline void __TBB_get_cpu_ctl_env ( tbb::internal::cpu_ctl_env* ctl ) {
__asm {
__asm mov __TBB_r(ax), ctl
__asm stmxcsr [__TBB_r(ax)]
__asm fstcw [__TBB_r(ax)+4]
}
}
inline void __TBB_set_cpu_ctl_env ( const tbb::internal::cpu_ctl_env* ctl ) {
__asm {
__asm mov __TBB_r(ax), ctl
__asm ldmxcsr [__TBB_r(ax)]
__asm fldcw [__TBB_r(ax)+4]
}
}
#else
extern "C" {
void __TBB_EXPORTED_FUNC __TBB_get_cpu_ctl_env ( tbb::internal::cpu_ctl_env* );
void __TBB_EXPORTED_FUNC __TBB_set_cpu_ctl_env ( const tbb::internal::cpu_ctl_env* );
}
#endif
namespace tbb {
namespace internal {
class cpu_ctl_env {
private:
int mxcsr;
short x87cw;
static const int MXCSR_CONTROL_MASK = ~0x3f; /* all except last six status bits */
public:
bool operator!=( const cpu_ctl_env& ctl ) const { return mxcsr != ctl.mxcsr || x87cw != ctl.x87cw; }
void get_env() {
__TBB_get_cpu_ctl_env( this );
mxcsr &= MXCSR_CONTROL_MASK;
}
void set_env() const { __TBB_set_cpu_ctl_env( this ); }
};
} // namespace internal
} // namespace tbb
#if !__TBB_WIN8UI_SUPPORT
extern "C" __declspec(dllimport) int __stdcall SwitchToThread( void );
#define __TBB_Yield() SwitchToThread()
#else
#include<thread>
#define __TBB_Yield() std::this_thread::yield()
#endif
#define __TBB_Pause(V) __TBB_machine_pause(V)
#define __TBB_Log2(V) __TBB_machine_lg(V)
#undef __TBB_r
extern "C" {
__int8 __TBB_EXPORTED_FUNC __TBB_machine_try_lock_elided (volatile void* ptr);
void __TBB_EXPORTED_FUNC __TBB_machine_unlock_elided (volatile void* ptr);
// 'pause' instruction aborts HLE/RTM transactions
#if __TBB_PAUSE_USE_INTRINSIC
inline static void __TBB_machine_try_lock_elided_cancel() { _mm_pause(); }
#else
inline static void __TBB_machine_try_lock_elided_cancel() { _asm pause; }
#endif
#if __TBB_TSX_INTRINSICS_PRESENT
#define __TBB_machine_is_in_transaction _xtest
#define __TBB_machine_begin_transaction _xbegin
#define __TBB_machine_end_transaction _xend
// The value (0xFF) below comes from the
// Intel(R) 64 and IA-32 Architectures Optimization Reference Manual 12.4.5 lock not free
#define __TBB_machine_transaction_conflict_abort() _xabort(0xFF)
#else
__int8 __TBB_EXPORTED_FUNC __TBB_machine_is_in_transaction();
unsigned __int32 __TBB_EXPORTED_FUNC __TBB_machine_begin_transaction();
void __TBB_EXPORTED_FUNC __TBB_machine_end_transaction();
void __TBB_EXPORTED_FUNC __TBB_machine_transaction_conflict_abort();
#endif /* __TBB_TSX_INTRINSICS_PRESENT */
}
#endif /* __TBB_machine_msvc_ia32_common_H */

View File

@@ -0,0 +1,203 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_sunos_sparc_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_sunos_sparc_H
#include <stdint.h>
#include <unistd.h>
#define __TBB_WORDSIZE 8
// Big endian is assumed for SPARC.
// While hardware may support page-specific bi-endianness, only big endian pages may be exposed to TBB
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
/** To those working on SPARC hardware. Consider relaxing acquire and release
consistency helpers to no-op (as this port covers TSO mode only). **/
#define __TBB_compiler_fence() __asm__ __volatile__ ("": : :"memory")
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#define __TBB_full_memory_fence() __asm__ __volatile__("membar #LoadLoad|#LoadStore|#StoreStore|#StoreLoad": : : "memory")
//--------------------------------------------------
// Compare and swap
//--------------------------------------------------
/**
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
* @param value value to assign *ptr to if *ptr==comparand
* @param comparand value to compare with *ptr
( @return value originally in memory at ptr, regardless of success
*/
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand ){
int32_t result;
__asm__ __volatile__(
"cas\t[%5],%4,%1"
: "=m"(*(int32_t *)ptr), "=r"(result)
: "m"(*(int32_t *)ptr), "1"(value), "r"(comparand), "r"(ptr)
: "memory");
return result;
}
/**
* Atomic CAS for 64 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
* @param value value to assign *ptr to if *ptr==comparand
* @param comparand value to compare with *ptr
( @return value originally in memory at ptr, regardless of success
*/
static inline int64_t __TBB_machine_cmpswp8(volatile void *ptr, int64_t value, int64_t comparand ){
int64_t result;
__asm__ __volatile__(
"casx\t[%5],%4,%1"
: "=m"(*(int64_t *)ptr), "=r"(result)
: "m"(*(int64_t *)ptr), "1"(value), "r"(comparand), "r"(ptr)
: "memory");
return result;
}
//---------------------------------------------------
// Fetch and add
//---------------------------------------------------
/**
* Atomic fetch and add for 32 bit values, in this case implemented by continuously checking success of atomicity
* @param ptr pointer to value to add addend to
* @param addened value to add to *ptr
* @return value at ptr before addened was added
*/
static inline int32_t __TBB_machine_fetchadd4(volatile void *ptr, int32_t addend){
int32_t result;
__asm__ __volatile__ (
"0:\t add\t %3, %4, %0\n" // do addition
"\t cas\t [%2], %3, %0\n" // cas to store result in memory
"\t cmp\t %3, %0\n" // check if value from memory is original
"\t bne,a,pn\t %%icc, 0b\n" // if not try again
"\t mov %0, %3\n" // use branch delay slot to move new value in memory to be added
: "=&r"(result), "=m"(*(int32_t *)ptr)
: "r"(ptr), "r"(*(int32_t *)ptr), "r"(addend), "m"(*(int32_t *)ptr)
: "ccr", "memory");
return result;
}
/**
* Atomic fetch and add for 64 bit values, in this case implemented by continuously checking success of atomicity
* @param ptr pointer to value to add addend to
* @param addened value to add to *ptr
* @return value at ptr before addened was added
*/
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend){
int64_t result;
__asm__ __volatile__ (
"0:\t add\t %3, %4, %0\n" // do addition
"\t casx\t [%2], %3, %0\n" // cas to store result in memory
"\t cmp\t %3, %0\n" // check if value from memory is original
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
"\t mov %0, %3\n" // use branch delay slot to move new value in memory to be added
: "=&r"(result), "=m"(*(int64_t *)ptr)
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(addend), "m"(*(int64_t *)ptr)
: "ccr", "memory");
return result;
}
//--------------------------------------------------------
// Logarithm (base two, integer)
//--------------------------------------------------------
static inline int64_t __TBB_machine_lg( uint64_t x ) {
__TBB_ASSERT(x, "__TBB_Log2(0) undefined");
uint64_t count;
// one hot encode
x |= (x >> 1);
x |= (x >> 2);
x |= (x >> 4);
x |= (x >> 8);
x |= (x >> 16);
x |= (x >> 32);
// count 1's
__asm__ ("popc %1, %0" : "=r"(count) : "r"(x) );
return count-1;
}
//--------------------------------------------------------
static inline void __TBB_machine_or( volatile void *ptr, uint64_t value ) {
__asm__ __volatile__ (
"0:\t or\t %2, %3, %%g1\n" // do operation
"\t casx\t [%1], %2, %%g1\n" // cas to store result in memory
"\t cmp\t %2, %%g1\n" // check if value from memory is original
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
"\t mov %%g1, %2\n" // use branch delay slot to move new value in memory to be added
: "=m"(*(int64_t *)ptr)
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(value), "m"(*(int64_t *)ptr)
: "ccr", "g1", "memory");
}
static inline void __TBB_machine_and( volatile void *ptr, uint64_t value ) {
__asm__ __volatile__ (
"0:\t and\t %2, %3, %%g1\n" // do operation
"\t casx\t [%1], %2, %%g1\n" // cas to store result in memory
"\t cmp\t %2, %%g1\n" // check if value from memory is original
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
"\t mov %%g1, %2\n" // use branch delay slot to move new value in memory to be added
: "=m"(*(int64_t *)ptr)
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(value), "m"(*(int64_t *)ptr)
: "ccr", "g1", "memory");
}
static inline void __TBB_machine_pause( int32_t delay ) {
// do nothing, inlined, doesn't matter
}
// put 0xff in memory location, return memory value,
// generic trylockbyte puts 0x01, however this is fine
// because all that matters is that 0 is unlocked
static inline bool __TBB_machine_trylockbyte(unsigned char &flag){
unsigned char result;
__asm__ __volatile__ (
"ldstub\t [%2], %0\n"
: "=r"(result), "=m"(flag)
: "r"(&flag), "m"(flag)
: "memory");
return result == 0;
}
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
#define __TBB_USE_GENERIC_FETCH_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
// Definition of other functions
#define __TBB_Pause(V) __TBB_machine_pause(V)
#define __TBB_Log2(V) __TBB_machine_lg(V)
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)

View File

@@ -0,0 +1,79 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#ifndef __TBB_machine_windows_api_H
#define __TBB_machine_windows_api_H
#if _WIN32 || _WIN64
#if _XBOX
#define NONET
#define NOD3D
#include <xtl.h>
#else // Assume "usual" Windows
#include <windows.h>
#endif // _XBOX
#if _WIN32_WINNT < 0x0600
// The following Windows API function is declared explicitly;
// otherwise it fails to compile by VS2005.
#if !defined(WINBASEAPI) || (_WIN32_WINNT < 0x0501 && _MSC_VER == 1400)
#define __TBB_WINBASEAPI extern "C"
#else
#define __TBB_WINBASEAPI WINBASEAPI
#endif
__TBB_WINBASEAPI BOOL WINAPI TryEnterCriticalSection( LPCRITICAL_SECTION );
__TBB_WINBASEAPI BOOL WINAPI InitializeCriticalSectionAndSpinCount( LPCRITICAL_SECTION, DWORD );
// Overloading WINBASEAPI macro and using local functions missing in Windows XP/2003
#define InitializeCriticalSectionEx inlineInitializeCriticalSectionEx
#define CreateSemaphoreEx inlineCreateSemaphoreEx
#define CreateEventEx inlineCreateEventEx
inline BOOL WINAPI inlineInitializeCriticalSectionEx( LPCRITICAL_SECTION lpCriticalSection, DWORD dwSpinCount, DWORD )
{
return InitializeCriticalSectionAndSpinCount( lpCriticalSection, dwSpinCount );
}
inline HANDLE WINAPI inlineCreateSemaphoreEx( LPSECURITY_ATTRIBUTES lpSemaphoreAttributes, LONG lInitialCount, LONG lMaximumCount, LPCTSTR lpName, DWORD, DWORD )
{
return CreateSemaphore( lpSemaphoreAttributes, lInitialCount, lMaximumCount, lpName );
}
inline HANDLE WINAPI inlineCreateEventEx( LPSECURITY_ATTRIBUTES lpEventAttributes, LPCTSTR lpName, DWORD dwFlags, DWORD )
{
BOOL manual_reset = dwFlags&0x00000001 ? TRUE : FALSE; // CREATE_EVENT_MANUAL_RESET
BOOL initial_set = dwFlags&0x00000002 ? TRUE : FALSE; // CREATE_EVENT_INITIAL_SET
return CreateEvent( lpEventAttributes, manual_reset, initial_set, lpName );
}
#endif
#if defined(RTL_SRWLOCK_INIT)
#ifndef __TBB_USE_SRWLOCK
// TODO: turn it on when bug 1952 will be fixed
#define __TBB_USE_SRWLOCK 0
#endif
#endif
#else
#error tbb/machine/windows_api.h should only be used for Windows based platforms
#endif // _WIN32 || _WIN64
#endif // __TBB_machine_windows_api_H

View File

@@ -0,0 +1,144 @@
/*
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
you can redistribute it and/or modify it under the terms of the GNU General Public License
version 2 as published by the Free Software Foundation. Threading Building Blocks is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details. You should have received a copy of
the GNU General Public License along with Threading Building Blocks; if not, write to the
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
As a special exception, you may use this file as part of a free software library without
restriction. Specifically, if other files instantiate templates or use macros or inline
functions from this file, or you compile this file and link it with other files to produce
an executable, this file does not by itself cause the resulting executable to be covered
by the GNU General Public License. This exception does not however invalidate any other
reasons why the executable file might be covered by the GNU General Public License.
*/
#if !defined(__TBB_machine_H) || defined(__TBB_machine_windows_ia32_H)
#error Do not #include this internal file directly; use public TBB headers instead.
#endif
#define __TBB_machine_windows_ia32_H
#include "msvc_ia32_common.h"
#define __TBB_WORDSIZE 4
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
#if __INTEL_COMPILER && (__INTEL_COMPILER < 1100)
#define __TBB_compiler_fence() __asm { __asm nop }
#define __TBB_full_memory_fence() __asm { __asm mfence }
#elif _MSC_VER >= 1300 || __INTEL_COMPILER
#pragma intrinsic(_ReadWriteBarrier)
#pragma intrinsic(_mm_mfence)
#define __TBB_compiler_fence() _ReadWriteBarrier()
#define __TBB_full_memory_fence() _mm_mfence()
#else
#error Unsupported compiler - need to define __TBB_{control,acquire,release}_consistency_helper to support it
#endif
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
// Workaround for overzealous compiler warnings in /Wp64 mode
#pragma warning (push)
#pragma warning (disable: 4244 4267)
#endif
extern "C" {
__int64 __TBB_EXPORTED_FUNC __TBB_machine_cmpswp8 (volatile void *ptr, __int64 value, __int64 comparand );
__int64 __TBB_EXPORTED_FUNC __TBB_machine_fetchadd8 (volatile void *ptr, __int64 addend );
__int64 __TBB_EXPORTED_FUNC __TBB_machine_fetchstore8 (volatile void *ptr, __int64 value );
void __TBB_EXPORTED_FUNC __TBB_machine_store8 (volatile void *ptr, __int64 value );
__int64 __TBB_EXPORTED_FUNC __TBB_machine_load8 (const volatile void *ptr);
}
//TODO: use _InterlockedXXX intrinsics as they available since VC 2005
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,U,A,C) \
static inline T __TBB_machine_cmpswp##S ( volatile void * ptr, U value, U comparand ) { \
T result; \
volatile T *p = (T *)ptr; \
__asm \
{ \
__asm mov edx, p \
__asm mov C , value \
__asm mov A , comparand \
__asm lock cmpxchg [edx], C \
__asm mov result, A \
} \
return result; \
} \
\
static inline T __TBB_machine_fetchadd##S ( volatile void * ptr, U addend ) { \
T result; \
volatile T *p = (T *)ptr; \
__asm \
{ \
__asm mov edx, p \
__asm mov A, addend \
__asm lock xadd [edx], A \
__asm mov result, A \
} \
return result; \
}\
\
static inline T __TBB_machine_fetchstore##S ( volatile void * ptr, U value ) { \
T result; \
volatile T *p = (T *)ptr; \
__asm \
{ \
__asm mov edx, p \
__asm mov A, value \
__asm lock xchg [edx], A \
__asm mov result, A \
} \
return result; \
}
__TBB_MACHINE_DEFINE_ATOMICS(1, __int8, __int8, al, cl)
__TBB_MACHINE_DEFINE_ATOMICS(2, __int16, __int16, ax, cx)
__TBB_MACHINE_DEFINE_ATOMICS(4, ptrdiff_t, ptrdiff_t, eax, ecx)
#undef __TBB_MACHINE_DEFINE_ATOMICS
static inline void __TBB_machine_OR( volatile void *operand, __int32 addend ) {
__asm
{
mov eax, addend
mov edx, [operand]
lock or [edx], eax
}
}
static inline void __TBB_machine_AND( volatile void *operand, __int32 addend ) {
__asm
{
mov eax, addend
mov edx, [operand]
lock and [edx], eax
}
}
#define __TBB_AtomicOR(P,V) __TBB_machine_OR(P,V)
#define __TBB_AtomicAND(P,V) __TBB_machine_AND(P,V)
//TODO: Check if it possible and profitable for IA-32 architecture on (Linux and Windows)
//to use of 64-bit load/store via floating point registers together with full fence
//for sequentially consistent load/store, instead of CAS.
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
#pragma warning (pop)
#endif // warnings 4244, 4267 are back

Some files were not shown because too many files have changed in this diff Show More