Lesson 35 - Get Compute Auth Token Working
This commit is contained in:
353
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/COPYING
vendored
Normal file
353
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/COPYING
vendored
Normal file
@@ -0,0 +1,353 @@
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
Version 2, June 1991
|
||||
|
||||
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
|
||||
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The licenses for most software are designed to take away your
|
||||
freedom to share and change it. By contrast, the GNU General Public
|
||||
License is intended to guarantee your freedom to share and change free
|
||||
software--to make sure the software is free for all its users. This
|
||||
General Public License applies to most of the Free Software
|
||||
Foundation's software and to any other program whose authors commit to
|
||||
using it. (Some other Free Software Foundation software is covered by
|
||||
the GNU Lesser General Public License instead.) You can apply it to
|
||||
your programs, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
this service if you wish), that you receive source code or can get it
|
||||
if you want it, that you can change the software or use pieces of it
|
||||
in new free programs; and that you know you can do these things.
|
||||
|
||||
To protect your rights, we need to make restrictions that forbid
|
||||
anyone to deny you these rights or to ask you to surrender the rights.
|
||||
These restrictions translate to certain responsibilities for you if you
|
||||
distribute copies of the software, or if you modify it.
|
||||
|
||||
For example, if you distribute copies of such a program, whether
|
||||
gratis or for a fee, you must give the recipients all the rights that
|
||||
you have. You must make sure that they, too, receive or can get the
|
||||
source code. And you must show them these terms so they know their
|
||||
rights.
|
||||
|
||||
We protect your rights with two steps: (1) copyright the software, and
|
||||
(2) offer you this license which gives you legal permission to copy,
|
||||
distribute and/or modify the software.
|
||||
|
||||
Also, for each author's protection and ours, we want to make certain
|
||||
that everyone understands that there is no warranty for this free
|
||||
software. If the software is modified by someone else and passed on, we
|
||||
want its recipients to know that what they have is not the original, so
|
||||
that any problems introduced by others will not reflect on the original
|
||||
authors' reputations.
|
||||
|
||||
Finally, any free program is threatened constantly by software
|
||||
patents. We wish to avoid the danger that redistributors of a free
|
||||
program will individually obtain patent licenses, in effect making the
|
||||
program proprietary. To prevent this, we have made it clear that any
|
||||
patent must be licensed for everyone's free use or not licensed at all.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||||
|
||||
0. This License applies to any program or other work which contains
|
||||
a notice placed by the copyright holder saying it may be distributed
|
||||
under the terms of this General Public License. The "Program", below,
|
||||
refers to any such program or work, and a "work based on the Program"
|
||||
means either the Program or any derivative work under copyright law:
|
||||
that is to say, a work containing the Program or a portion of it,
|
||||
either verbatim or with modifications and/or translated into another
|
||||
language. (Hereinafter, translation is included without limitation in
|
||||
the term "modification".) Each licensee is addressed as "you".
|
||||
|
||||
Activities other than copying, distribution and modification are not
|
||||
covered by this License; they are outside its scope. The act of
|
||||
running the Program is not restricted, and the output from the Program
|
||||
is covered only if its contents constitute a work based on the
|
||||
Program (independent of having been made by running the Program).
|
||||
Whether that is true depends on what the Program does.
|
||||
|
||||
1. You may copy and distribute verbatim copies of the Program's
|
||||
source code as you receive it, in any medium, provided that you
|
||||
conspicuously and appropriately publish on each copy an appropriate
|
||||
copyright notice and disclaimer of warranty; keep intact all the
|
||||
notices that refer to this License and to the absence of any warranty;
|
||||
and give any other recipients of the Program a copy of this License
|
||||
along with the Program.
|
||||
|
||||
You may charge a fee for the physical act of transferring a copy, and
|
||||
you may at your option offer warranty protection in exchange for a fee.
|
||||
|
||||
2. You may modify your copy or copies of the Program or any portion
|
||||
of it, thus forming a work based on the Program, and copy and
|
||||
distribute such modifications or work under the terms of Section 1
|
||||
above, provided that you also meet all of these conditions:
|
||||
|
||||
a) You must cause the modified files to carry prominent notices
|
||||
stating that you changed the files and the date of any change.
|
||||
|
||||
b) You must cause any work that you distribute or publish, that in
|
||||
whole or in part contains or is derived from the Program or any
|
||||
part thereof, to be licensed as a whole at no charge to all third
|
||||
parties under the terms of this License.
|
||||
|
||||
c) If the modified program normally reads commands interactively
|
||||
when run, you must cause it, when started running for such
|
||||
interactive use in the most ordinary way, to print or display an
|
||||
announcement including an appropriate copyright notice and a
|
||||
notice that there is no warranty (or else, saying that you provide
|
||||
a warranty) and that users may redistribute the program under
|
||||
these conditions, and telling the user how to view a copy of this
|
||||
License. (Exception: if the Program itself is interactive but
|
||||
does not normally print such an announcement, your work based on
|
||||
the Program is not required to print an announcement.)
|
||||
|
||||
These requirements apply to the modified work as a whole. If
|
||||
identifiable sections of that work are not derived from the Program,
|
||||
and can be reasonably considered independent and separate works in
|
||||
themselves, then this License, and its terms, do not apply to those
|
||||
sections when you distribute them as separate works. But when you
|
||||
distribute the same sections as part of a whole which is a work based
|
||||
on the Program, the distribution of the whole must be on the terms of
|
||||
this License, whose permissions for other licensees extend to the
|
||||
entire whole, and thus to each and every part regardless of who wrote it.
|
||||
|
||||
Thus, it is not the intent of this section to claim rights or contest
|
||||
your rights to work written entirely by you; rather, the intent is to
|
||||
exercise the right to control the distribution of derivative or
|
||||
collective works based on the Program.
|
||||
|
||||
In addition, mere aggregation of another work not based on the Program
|
||||
with the Program (or with a work based on the Program) on a volume of
|
||||
a storage or distribution medium does not bring the other work under
|
||||
the scope of this License.
|
||||
|
||||
3. You may copy and distribute the Program (or a work based on it,
|
||||
under Section 2) in object code or executable form under the terms of
|
||||
Sections 1 and 2 above provided that you also do one of the following:
|
||||
|
||||
a) Accompany it with the complete corresponding machine-readable
|
||||
source code, which must be distributed under the terms of Sections
|
||||
1 and 2 above on a medium customarily used for software interchange; or,
|
||||
|
||||
b) Accompany it with a written offer, valid for at least three
|
||||
years, to give any third party, for a charge no more than your
|
||||
cost of physically performing source distribution, a complete
|
||||
machine-readable copy of the corresponding source code, to be
|
||||
distributed under the terms of Sections 1 and 2 above on a medium
|
||||
customarily used for software interchange; or,
|
||||
|
||||
c) Accompany it with the information you received as to the offer
|
||||
to distribute corresponding source code. (This alternative is
|
||||
allowed only for noncommercial distribution and only if you
|
||||
received the program in object code or executable form with such
|
||||
an offer, in accord with Subsection b above.)
|
||||
|
||||
The source code for a work means the preferred form of the work for
|
||||
making modifications to it. For an executable work, complete source
|
||||
code means all the source code for all modules it contains, plus any
|
||||
associated interface definition files, plus the scripts used to
|
||||
control compilation and installation of the executable. However, as a
|
||||
special exception, the source code distributed need not include
|
||||
anything that is normally distributed (in either source or binary
|
||||
form) with the major components (compiler, kernel, and so on) of the
|
||||
operating system on which the executable runs, unless that component
|
||||
itself accompanies the executable.
|
||||
|
||||
If distribution of executable or object code is made by offering
|
||||
access to copy from a designated place, then offering equivalent
|
||||
access to copy the source code from the same place counts as
|
||||
distribution of the source code, even though third parties are not
|
||||
compelled to copy the source along with the object code.
|
||||
|
||||
4. You may not copy, modify, sublicense, or distribute the Program
|
||||
except as expressly provided under this License. Any attempt
|
||||
otherwise to copy, modify, sublicense or distribute the Program is
|
||||
void, and will automatically terminate your rights under this License.
|
||||
However, parties who have received copies, or rights, from you under
|
||||
this License will not have their licenses terminated so long as such
|
||||
parties remain in full compliance.
|
||||
|
||||
5. You are not required to accept this License, since you have not
|
||||
signed it. However, nothing else grants you permission to modify or
|
||||
distribute the Program or its derivative works. These actions are
|
||||
prohibited by law if you do not accept this License. Therefore, by
|
||||
modifying or distributing the Program (or any work based on the
|
||||
Program), you indicate your acceptance of this License to do so, and
|
||||
all its terms and conditions for copying, distributing or modifying
|
||||
the Program or works based on it.
|
||||
|
||||
6. Each time you redistribute the Program (or any work based on the
|
||||
Program), the recipient automatically receives a license from the
|
||||
original licensor to copy, distribute or modify the Program subject to
|
||||
these terms and conditions. You may not impose any further
|
||||
restrictions on the recipients' exercise of the rights granted herein.
|
||||
You are not responsible for enforcing compliance by third parties to
|
||||
this License.
|
||||
|
||||
7. If, as a consequence of a court judgment or allegation of patent
|
||||
infringement or for any other reason (not limited to patent issues),
|
||||
conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot
|
||||
distribute so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you
|
||||
may not distribute the Program at all. For example, if a patent
|
||||
license would not permit royalty-free redistribution of the Program by
|
||||
all those who receive copies directly or indirectly through you, then
|
||||
the only way you could satisfy both it and this License would be to
|
||||
refrain entirely from distribution of the Program.
|
||||
|
||||
If any portion of this section is held invalid or unenforceable under
|
||||
any particular circumstance, the balance of the section is intended to
|
||||
apply and the section as a whole is intended to apply in other
|
||||
circumstances.
|
||||
|
||||
It is not the purpose of this section to induce you to infringe any
|
||||
patents or other property right claims or to contest validity of any
|
||||
such claims; this section has the sole purpose of protecting the
|
||||
integrity of the free software distribution system, which is
|
||||
implemented by public license practices. Many people have made
|
||||
generous contributions to the wide range of software distributed
|
||||
through that system in reliance on consistent application of that
|
||||
system; it is up to the author/donor to decide if he or she is willing
|
||||
to distribute software through any other system and a licensee cannot
|
||||
impose that choice.
|
||||
|
||||
This section is intended to make thoroughly clear what is believed to
|
||||
be a consequence of the rest of this License.
|
||||
|
||||
8. If the distribution and/or use of the Program is restricted in
|
||||
certain countries either by patents or by copyrighted interfaces, the
|
||||
original copyright holder who places the Program under this License
|
||||
may add an explicit geographical distribution limitation excluding
|
||||
those countries, so that distribution is permitted only in or among
|
||||
countries not thus excluded. In such case, this License incorporates
|
||||
the limitation as if written in the body of this License.
|
||||
|
||||
9. The Free Software Foundation may publish revised and/or new versions
|
||||
of the General Public License from time to time. Such new versions will
|
||||
be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the Program
|
||||
specifies a version number of this License which applies to it and "any
|
||||
later version", you have the option of following the terms and conditions
|
||||
either of that version or of any later version published by the Free
|
||||
Software Foundation. If the Program does not specify a version number of
|
||||
this License, you may choose any version ever published by the Free Software
|
||||
Foundation.
|
||||
|
||||
10. If you wish to incorporate parts of the Program into other free
|
||||
programs whose distribution conditions are different, write to the author
|
||||
to ask for permission. For software which is copyrighted by the Free
|
||||
Software Foundation, write to the Free Software Foundation; we sometimes
|
||||
make exceptions for this. Our decision will be guided by the two goals
|
||||
of preserving the free status of all derivatives of our free software and
|
||||
of promoting the sharing and reuse of software generally.
|
||||
|
||||
NO WARRANTY
|
||||
|
||||
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
|
||||
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
|
||||
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
|
||||
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
|
||||
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
|
||||
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
|
||||
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
|
||||
REPAIR OR CORRECTION.
|
||||
|
||||
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
|
||||
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
|
||||
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
|
||||
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
|
||||
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
|
||||
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
|
||||
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
|
||||
POSSIBILITY OF SUCH DAMAGES.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
convey the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software; you can redistribute it and/or modify
|
||||
it under the terms of the GNU General Public License as published by
|
||||
the Free Software Foundation; either version 2 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along
|
||||
with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program is interactive, make it output a short notice like this
|
||||
when it starts in an interactive mode:
|
||||
|
||||
Gnomovision version 69, Copyright (C) year name of author
|
||||
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
This is free software, and you are welcome to redistribute it
|
||||
under certain conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, the commands you use may
|
||||
be called something other than `show w' and `show c'; they could even be
|
||||
mouse-clicks or menu items--whatever suits your program.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or your
|
||||
school, if any, to sign a "copyright disclaimer" for the program, if
|
||||
necessary. Here is a sample; alter the names:
|
||||
|
||||
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
|
||||
`Gnomovision' (which makes passes at compilers) written by James Hacker.
|
||||
|
||||
<signature of Ty Coon>, 1 April 1989
|
||||
Ty Coon, President of Vice
|
||||
|
||||
This General Public License does not permit incorporating your program into
|
||||
proprietary programs. If your program is a subroutine library, you may
|
||||
consider it more useful to permit linking proprietary applications with the
|
||||
library. If this is what you want to do, use the GNU Lesser General
|
||||
Public License instead of this License.
|
||||
---------------- END OF Gnu General Public License ----------------
|
||||
|
||||
The source code of Threading Building Blocks is distributed under version 2
|
||||
of the GNU General Public License, with the so-called "runtime exception,"
|
||||
as follows (or see any header or implementation file):
|
||||
|
||||
As a special exception, you may use this file as part of a free software
|
||||
library without restriction. Specifically, if other files instantiate
|
||||
templates or use macros or inline functions from this file, or you compile
|
||||
this file and link it with other files to produce an executable, this
|
||||
file does not by itself cause the resulting executable to be covered by
|
||||
the GNU General Public License. This exception does not however
|
||||
invalidate any other reasons why the executable file might be covered by
|
||||
the GNU General Public License.
|
||||
@@ -0,0 +1,3 @@
|
||||
This is a partial copy of the Intel TBB open source version.
|
||||
The version taken is 4.3, obtained from https://www.threadingbuildingblocks.org/download
|
||||
The files in this directory consist of the files taken from src/tbb and include/tbb
|
||||
202
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/aggregator.h
vendored
Normal file
202
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/aggregator.h
vendored
Normal file
@@ -0,0 +1,202 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__aggregator_H
|
||||
#define __TBB__aggregator_H
|
||||
|
||||
#if !TBB_PREVIEW_AGGREGATOR
|
||||
#error Set TBB_PREVIEW_AGGREGATOR before including aggregator.h
|
||||
#endif
|
||||
|
||||
#include "atomic.h"
|
||||
#include "tbb_profiling.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace interface6 {
|
||||
|
||||
using namespace tbb::internal;
|
||||
|
||||
class aggregator_operation {
|
||||
template<typename handler_type> friend class aggregator_ext;
|
||||
uintptr_t status;
|
||||
aggregator_operation* my_next;
|
||||
public:
|
||||
enum aggregator_operation_status { agg_waiting=0, agg_finished };
|
||||
aggregator_operation() : status(agg_waiting), my_next(NULL) {}
|
||||
/// Call start before handling this operation
|
||||
void start() { call_itt_notify(acquired, &status); }
|
||||
/// Call finish when done handling this operation
|
||||
/** The operation will be released to its originating thread, and possibly deleted. */
|
||||
void finish() { itt_store_word_with_release(status, uintptr_t(agg_finished)); }
|
||||
aggregator_operation* next() { return itt_hide_load_word(my_next);}
|
||||
void set_next(aggregator_operation* n) { itt_hide_store_word(my_next, n); }
|
||||
};
|
||||
|
||||
namespace internal {
|
||||
|
||||
class basic_operation_base : public aggregator_operation {
|
||||
friend class basic_handler;
|
||||
virtual void apply_body() = 0;
|
||||
public:
|
||||
basic_operation_base() : aggregator_operation() {}
|
||||
virtual ~basic_operation_base() {}
|
||||
};
|
||||
|
||||
template<typename Body>
|
||||
class basic_operation : public basic_operation_base, no_assign {
|
||||
const Body& my_body;
|
||||
/*override*/ void apply_body() { my_body(); }
|
||||
public:
|
||||
basic_operation(const Body& b) : basic_operation_base(), my_body(b) {}
|
||||
};
|
||||
|
||||
class basic_handler {
|
||||
public:
|
||||
basic_handler() {}
|
||||
void operator()(aggregator_operation* op_list) const {
|
||||
while (op_list) {
|
||||
// ITT note: &(op_list->status) tag is used to cover accesses to the operation data.
|
||||
// The executing thread "acquires" the tag (see start()) and then performs
|
||||
// the associated operation w/o triggering a race condition diagnostics.
|
||||
// A thread that created the operation is waiting for its status (see execute_impl()),
|
||||
// so when this thread is done with the operation, it will "release" the tag
|
||||
// and update the status (see finish()) to give control back to the waiting thread.
|
||||
basic_operation_base& request = static_cast<basic_operation_base&>(*op_list);
|
||||
// IMPORTANT: need to advance op_list to op_list->next() before calling request.finish()
|
||||
op_list = op_list->next();
|
||||
request.start();
|
||||
request.apply_body();
|
||||
request.finish();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace internal
|
||||
|
||||
//! Aggregator base class and expert interface
|
||||
/** An aggregator for collecting operations coming from multiple sources and executing
|
||||
them serially on a single thread. */
|
||||
template <typename handler_type>
|
||||
class aggregator_ext : tbb::internal::no_copy {
|
||||
public:
|
||||
aggregator_ext(const handler_type& h) : handler_busy(0), handle_operations(h) { mailbox = NULL; }
|
||||
|
||||
//! EXPERT INTERFACE: Enter a user-made operation into the aggregator's mailbox.
|
||||
/** Details of user-made operations must be handled by user-provided handler */
|
||||
void process(aggregator_operation *op) { execute_impl(*op); }
|
||||
|
||||
protected:
|
||||
/** Place operation in mailbox, then either handle mailbox or wait for the operation
|
||||
to be completed by a different thread. */
|
||||
void execute_impl(aggregator_operation& op) {
|
||||
aggregator_operation* res;
|
||||
|
||||
// ITT note: &(op.status) tag is used to cover accesses to this operation. This
|
||||
// thread has created the operation, and now releases it so that the handler
|
||||
// thread may handle the associated operation w/o triggering a race condition;
|
||||
// thus this tag will be acquired just before the operation is handled in the
|
||||
// handle_operations functor.
|
||||
call_itt_notify(releasing, &(op.status));
|
||||
// insert the operation in the queue
|
||||
do {
|
||||
// ITT may flag the following line as a race; it is a false positive:
|
||||
// This is an atomic read; we don't provide itt_hide_load_word for atomics
|
||||
op.my_next = res = mailbox; // NOT A RACE
|
||||
} while (mailbox.compare_and_swap(&op, res) != res);
|
||||
if (!res) { // first in the list; handle the operations
|
||||
// ITT note: &mailbox tag covers access to the handler_busy flag, which this
|
||||
// waiting handler thread will try to set before entering handle_operations.
|
||||
call_itt_notify(acquired, &mailbox);
|
||||
start_handle_operations();
|
||||
__TBB_ASSERT(op.status, NULL);
|
||||
}
|
||||
else { // not first; wait for op to be ready
|
||||
call_itt_notify(prepare, &(op.status));
|
||||
spin_wait_while_eq(op.status, uintptr_t(aggregator_operation::agg_waiting));
|
||||
itt_load_word_with_acquire(op.status);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
private:
|
||||
//! An atomically updated list (aka mailbox) of aggregator_operations
|
||||
atomic<aggregator_operation *> mailbox;
|
||||
|
||||
//! Controls thread access to handle_operations
|
||||
/** Behaves as boolean flag where 0=false, 1=true */
|
||||
uintptr_t handler_busy;
|
||||
|
||||
handler_type handle_operations;
|
||||
|
||||
//! Trigger the handling of operations when the handler is free
|
||||
void start_handle_operations() {
|
||||
aggregator_operation *pending_operations;
|
||||
|
||||
// ITT note: &handler_busy tag covers access to mailbox as it is passed
|
||||
// between active and waiting handlers. Below, the waiting handler waits until
|
||||
// the active handler releases, and the waiting handler acquires &handler_busy as
|
||||
// it becomes the active_handler. The release point is at the end of this
|
||||
// function, when all operations in mailbox have been handled by the
|
||||
// owner of this aggregator.
|
||||
call_itt_notify(prepare, &handler_busy);
|
||||
// get handler_busy: only one thread can possibly spin here at a time
|
||||
spin_wait_until_eq(handler_busy, uintptr_t(0));
|
||||
call_itt_notify(acquired, &handler_busy);
|
||||
// acquire fence not necessary here due to causality rule and surrounding atomics
|
||||
__TBB_store_with_release(handler_busy, uintptr_t(1));
|
||||
|
||||
// ITT note: &mailbox tag covers access to the handler_busy flag itself.
|
||||
// Capturing the state of the mailbox signifies that handler_busy has been
|
||||
// set and a new active handler will now process that list's operations.
|
||||
call_itt_notify(releasing, &mailbox);
|
||||
// grab pending_operations
|
||||
pending_operations = mailbox.fetch_and_store(NULL);
|
||||
|
||||
// handle all the operations
|
||||
handle_operations(pending_operations);
|
||||
|
||||
// release the handler
|
||||
itt_store_word_with_release(handler_busy, uintptr_t(0));
|
||||
}
|
||||
};
|
||||
|
||||
//! Basic aggregator interface
|
||||
class aggregator : private aggregator_ext<internal::basic_handler> {
|
||||
public:
|
||||
aggregator() : aggregator_ext<internal::basic_handler>(internal::basic_handler()) {}
|
||||
//! BASIC INTERFACE: Enter a function for exclusive execution by the aggregator.
|
||||
/** The calling thread stores the function object in a basic_operation and
|
||||
places the operation in the aggregator's mailbox */
|
||||
template<typename Body>
|
||||
void execute(const Body& b) {
|
||||
internal::basic_operation<Body> op(b);
|
||||
this->execute_impl(op);
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace interface6
|
||||
|
||||
using interface6::aggregator;
|
||||
using interface6::aggregator_ext;
|
||||
using interface6::aggregator_operation;
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif // __TBB__aggregator_H
|
||||
@@ -0,0 +1,47 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_aligned_space_H
|
||||
#define __TBB_aligned_space_H
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
#include "tbb_machine.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
//! Block of space aligned sufficiently to construct an array T with N elements.
|
||||
/** The elements are not constructed or destroyed by this class.
|
||||
@ingroup memory_allocation */
|
||||
template<typename T,size_t N=1>
|
||||
class aligned_space {
|
||||
private:
|
||||
typedef __TBB_TypeWithAlignmentAtLeastAsStrict(T) element_type;
|
||||
element_type array[(sizeof(T)*N+sizeof(element_type)-1)/sizeof(element_type)];
|
||||
public:
|
||||
//! Pointer to beginning of array
|
||||
T* begin() {return internal::punned_cast<T*>(this);}
|
||||
|
||||
//! Pointer to one past last element in array.
|
||||
T* end() {return begin()+N;}
|
||||
};
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_aligned_space_H */
|
||||
867
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/arena.cpp
vendored
Normal file
867
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/arena.cpp
vendored
Normal file
@@ -0,0 +1,867 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "scheduler.h"
|
||||
#include "governor.h"
|
||||
#include "arena.h"
|
||||
#include "itt_notify.h"
|
||||
#include "semaphore.h"
|
||||
|
||||
#include <functional>
|
||||
|
||||
#if __TBB_STATISTICS_STDOUT
|
||||
#include <cstdio>
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
void arena::process( generic_scheduler& s ) {
|
||||
__TBB_ASSERT( is_alive(my_guard), NULL );
|
||||
__TBB_ASSERT( governor::is_set(&s), NULL );
|
||||
__TBB_ASSERT( !s.my_innermost_running_task, NULL );
|
||||
__TBB_ASSERT( !s.my_dispatching_task, NULL );
|
||||
|
||||
__TBB_ASSERT( my_num_slots != 1, NULL );
|
||||
// Start search for an empty slot from the one we occupied the last time
|
||||
unsigned index = s.my_arena_index < my_num_slots ? s.my_arena_index : s.my_random.get() % (my_num_slots - 1) + 1,
|
||||
end = index;
|
||||
__TBB_ASSERT( index != 0, "A worker cannot occupy slot 0" );
|
||||
__TBB_ASSERT( index < my_num_slots, NULL );
|
||||
|
||||
// Find a vacant slot
|
||||
for ( ;; ) {
|
||||
if ( !my_slots[index].my_scheduler && as_atomic(my_slots[index].my_scheduler).compare_and_swap(&s, NULL ) == NULL )
|
||||
break;
|
||||
if ( ++index == my_num_slots )
|
||||
index = 1;
|
||||
if ( index == end ) {
|
||||
// Likely this arena is already saturated
|
||||
goto quit;
|
||||
}
|
||||
}
|
||||
ITT_NOTIFY(sync_acquired, my_slots + index);
|
||||
s.my_arena = this;
|
||||
s.my_arena_index = index;
|
||||
s.my_arena_slot = my_slots + index;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
s.my_local_reload_epoch = *s.my_ref_reload_epoch;
|
||||
__TBB_ASSERT( !s.my_offloaded_tasks, NULL );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
s.attach_mailbox( affinity_id(index+1) );
|
||||
|
||||
s.my_arena_slot->hint_for_pop = index; // initial value for round-robin
|
||||
|
||||
#if !__TBB_FP_CONTEXT
|
||||
my_cpu_ctl_env.set_env();
|
||||
#endif
|
||||
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_ASSERT( !s.my_last_local_observer, "There cannot be notified local observers when entering arena" );
|
||||
my_observers.notify_entry_observers( s.my_last_local_observer, /*worker=*/true );
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
|
||||
atomic_update( my_limit, index + 1, std::less<unsigned>() );
|
||||
|
||||
for ( ;; ) {
|
||||
// Try to steal a task.
|
||||
// Passing reference count is technically unnecessary in this context,
|
||||
// but omitting it here would add checks inside the function.
|
||||
__TBB_ASSERT( is_alive(my_guard), NULL );
|
||||
task* t = s.receive_or_steal_task( s.my_dummy_task->prefix().ref_count, /*return_if_no_work=*/true );
|
||||
if (t) {
|
||||
// A side effect of receive_or_steal_task is that my_innermost_running_task can be set.
|
||||
// But for the outermost dispatch loop of a worker it has to be NULL.
|
||||
s.my_innermost_running_task = NULL;
|
||||
__TBB_ASSERT( !s.my_dispatching_task, NULL );
|
||||
s.local_wait_for_all(*s.my_dummy_task,t);
|
||||
}
|
||||
__TBB_ASSERT ( __TBB_load_relaxed(s.my_arena_slot->head) == __TBB_load_relaxed(s.my_arena_slot->tail),
|
||||
"Worker cannot leave arena while its task pool is not empty" );
|
||||
__TBB_ASSERT( s.my_arena_slot->task_pool == EmptyTaskPool, "Empty task pool is not marked appropriately" );
|
||||
// This check prevents relinquishing more than necessary workers because
|
||||
// of the non-atomicity of the decision making procedure
|
||||
if (num_workers_active() > my_num_workers_allotted)
|
||||
break;
|
||||
}
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
my_observers.notify_exit_observers( s.my_last_local_observer, /*worker=*/true );
|
||||
s.my_last_local_observer = NULL;
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( s.my_offloaded_tasks )
|
||||
orphan_offloaded_tasks( s );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
#if __TBB_STATISTICS
|
||||
++s.my_counters.arena_roundtrips;
|
||||
*my_slots[index].my_counters += s.my_counters;
|
||||
s.my_counters.reset();
|
||||
#endif /* __TBB_STATISTICS */
|
||||
__TBB_store_with_release( my_slots[index].my_scheduler, (generic_scheduler*)NULL );
|
||||
s.my_arena_slot = 0; // detached from slot
|
||||
s.my_inbox.detach();
|
||||
__TBB_ASSERT( s.my_inbox.is_idle_state(true), NULL );
|
||||
__TBB_ASSERT( !s.my_innermost_running_task, NULL );
|
||||
__TBB_ASSERT( !s.my_dispatching_task, NULL );
|
||||
__TBB_ASSERT( is_alive(my_guard), NULL );
|
||||
quit:
|
||||
// In contrast to earlier versions of TBB (before 3.0 U5) now it is possible
|
||||
// that arena may be temporarily left unpopulated by threads. See comments in
|
||||
// arena::on_thread_leaving() for more details.
|
||||
#if !__TBB_TRACK_PRIORITY_LEVEL_SATURATION
|
||||
on_thread_leaving</*is_master*/false>();
|
||||
#endif /* !__TBB_TRACK_PRIORITY_LEVEL_SATURATION */
|
||||
}
|
||||
|
||||
arena::arena ( market& m, unsigned max_num_workers ) {
|
||||
__TBB_ASSERT( !my_guard, "improperly allocated arena?" );
|
||||
__TBB_ASSERT( sizeof(my_slots[0]) % NFS_GetLineSize()==0, "arena::slot size not multiple of cache line size" );
|
||||
__TBB_ASSERT( (uintptr_t)this % NFS_GetLineSize()==0, "arena misaligned" );
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_ASSERT( !my_reload_epoch && !my_orphaned_tasks && !my_skipped_fifo_priority, "New arena object is not zeroed" );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
my_market = &m;
|
||||
my_limit = 1;
|
||||
// Two slots are mandatory: for the master, and for 1 worker (required to support starvation resistant tasks).
|
||||
my_num_slots = num_slots_to_reserve(max_num_workers);
|
||||
my_max_num_workers = max_num_workers;
|
||||
my_references = 1; // accounts for the master
|
||||
#if __TBB_TASK_PRIORITY
|
||||
my_bottom_priority = my_top_priority = normalized_normal_priority;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
my_aba_epoch = m.my_arenas_aba_epoch;
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
my_observers.my_arena = this;
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_ASSERT ( my_max_num_workers < my_num_slots, NULL );
|
||||
// Construct slots. Mark internal synchronization elements for the tools.
|
||||
for( unsigned i = 0; i < my_num_slots; ++i ) {
|
||||
__TBB_ASSERT( !my_slots[i].my_scheduler && !my_slots[i].task_pool, NULL );
|
||||
__TBB_ASSERT( !my_slots[i].task_pool_ptr, NULL );
|
||||
__TBB_ASSERT( !my_slots[i].my_task_pool_size, NULL );
|
||||
ITT_SYNC_CREATE(my_slots + i, SyncType_Scheduler, SyncObj_WorkerTaskPool);
|
||||
mailbox(i+1).construct();
|
||||
ITT_SYNC_CREATE(&mailbox(i+1), SyncType_Scheduler, SyncObj_Mailbox);
|
||||
my_slots[i].hint_for_pop = i;
|
||||
#if __TBB_STATISTICS
|
||||
my_slots[i].my_counters = new ( NFS_Allocate(1, sizeof(statistics_counters), NULL) ) statistics_counters;
|
||||
#endif /* __TBB_STATISTICS */
|
||||
}
|
||||
#if __TBB_TASK_PRIORITY
|
||||
for ( intptr_t i = 0; i < num_priority_levels; ++i ) {
|
||||
my_task_stream[i].initialize(my_num_slots);
|
||||
ITT_SYNC_CREATE(my_task_stream + i, SyncType_Scheduler, SyncObj_TaskStream);
|
||||
}
|
||||
#else /* !__TBB_TASK_PRIORITY */
|
||||
my_task_stream.initialize(my_num_slots);
|
||||
ITT_SYNC_CREATE(&my_task_stream, SyncType_Scheduler, SyncObj_TaskStream);
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
my_mandatory_concurrency = false;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
// Context to be used by root tasks by default (if the user has not specified one).
|
||||
// The arena's context should not capture fp settings for the sake of backward compatibility.
|
||||
my_default_ctx =
|
||||
new ( NFS_Allocate(1, sizeof(task_group_context), NULL) ) task_group_context(task_group_context::isolated, task_group_context::default_traits);
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
#if __TBB_FP_CONTEXT
|
||||
my_default_ctx->capture_fp_settings();
|
||||
#else
|
||||
my_cpu_ctl_env.get_env();
|
||||
#endif
|
||||
}
|
||||
|
||||
arena& arena::allocate_arena( market& m, unsigned max_num_workers ) {
|
||||
__TBB_ASSERT( sizeof(base_type) + sizeof(arena_slot) == sizeof(arena), "All arena data fields must go to arena_base" );
|
||||
__TBB_ASSERT( sizeof(base_type) % NFS_GetLineSize() == 0, "arena slots area misaligned: wrong padding" );
|
||||
__TBB_ASSERT( sizeof(mail_outbox) == NFS_MaxLineSize, "Mailbox padding is wrong" );
|
||||
size_t n = allocation_size(max_num_workers);
|
||||
unsigned char* storage = (unsigned char*)NFS_Allocate( 1, n, NULL );
|
||||
// Zero all slots to indicate that they are empty
|
||||
memset( storage, 0, n );
|
||||
return *new( storage + num_slots_to_reserve(max_num_workers) * sizeof(mail_outbox) ) arena(m, max_num_workers);
|
||||
}
|
||||
|
||||
void arena::free_arena () {
|
||||
__TBB_ASSERT( is_alive(my_guard), NULL );
|
||||
__TBB_ASSERT( !my_references, "There are threads in the dying arena" );
|
||||
__TBB_ASSERT( !my_num_workers_requested && !my_num_workers_allotted, "Dying arena requests workers" );
|
||||
__TBB_ASSERT( my_pool_state == SNAPSHOT_EMPTY || !my_max_num_workers, "Inconsistent state of a dying arena" );
|
||||
#if !__TBB_STATISTICS_EARLY_DUMP
|
||||
GATHER_STATISTIC( dump_arena_statistics() );
|
||||
#endif
|
||||
poison_value( my_guard );
|
||||
intptr_t drained = 0;
|
||||
for ( unsigned i = 0; i < my_num_slots; ++i ) {
|
||||
__TBB_ASSERT( !my_slots[i].my_scheduler, "arena slot is not empty" );
|
||||
#if !__TBB_TASK_ARENA
|
||||
__TBB_ASSERT( my_slots[i].task_pool == EmptyTaskPool, NULL );
|
||||
#else
|
||||
//TODO: understand the assertion and modify
|
||||
#endif
|
||||
__TBB_ASSERT( my_slots[i].head == my_slots[i].tail, NULL ); // TODO: replace by is_quiescent_local_task_pool_empty
|
||||
my_slots[i].free_task_pool();
|
||||
#if __TBB_STATISTICS
|
||||
NFS_Free( my_slots[i].my_counters );
|
||||
#endif /* __TBB_STATISTICS */
|
||||
drained += mailbox(i+1).drain();
|
||||
}
|
||||
#if __TBB_TASK_PRIORITY && TBB_USE_ASSERT
|
||||
for ( intptr_t i = 0; i < num_priority_levels; ++i )
|
||||
__TBB_ASSERT(my_task_stream[i].empty() && my_task_stream[i].drain()==0, "Not all enqueued tasks were executed");
|
||||
#elif !__TBB_TASK_PRIORITY
|
||||
__TBB_ASSERT(my_task_stream.empty() && my_task_stream.drain()==0, "Not all enqueued tasks were executed");
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
#if __TBB_COUNT_TASK_NODES
|
||||
my_market->update_task_node_count( -drained );
|
||||
#endif /* __TBB_COUNT_TASK_NODES */
|
||||
my_market->release();
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_ASSERT( my_default_ctx, "Master thread never entered the arena?" );
|
||||
my_default_ctx->~task_group_context();
|
||||
NFS_Free(my_default_ctx);
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
if ( !my_observers.empty() )
|
||||
my_observers.clear();
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
void* storage = &mailbox(my_num_slots);
|
||||
__TBB_ASSERT( my_references == 0, NULL );
|
||||
__TBB_ASSERT( my_pool_state == SNAPSHOT_EMPTY || !my_max_num_workers, NULL );
|
||||
this->~arena();
|
||||
#if TBB_USE_ASSERT > 1
|
||||
memset( storage, 0, allocation_size(my_max_num_workers) );
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
NFS_Free( storage );
|
||||
}
|
||||
|
||||
#if __TBB_STATISTICS
|
||||
void arena::dump_arena_statistics () {
|
||||
statistics_counters total;
|
||||
for( unsigned i = 0; i < my_num_slots; ++i ) {
|
||||
#if __TBB_STATISTICS_EARLY_DUMP
|
||||
generic_scheduler* s = my_slots[i].my_scheduler;
|
||||
if ( s )
|
||||
*my_slots[i].my_counters += s->my_counters;
|
||||
#else
|
||||
__TBB_ASSERT( !my_slots[i].my_scheduler, NULL );
|
||||
#endif
|
||||
if ( i != 0 ) {
|
||||
total += *my_slots[i].my_counters;
|
||||
dump_statistics( *my_slots[i].my_counters, i );
|
||||
}
|
||||
}
|
||||
dump_statistics( *my_slots[0].my_counters, 0 );
|
||||
#if __TBB_STATISTICS_STDOUT
|
||||
#if !__TBB_STATISTICS_TOTALS_ONLY
|
||||
printf( "----------------------------------------------\n" );
|
||||
#endif
|
||||
dump_statistics( total, workers_counters_total );
|
||||
total += *my_slots[0].my_counters;
|
||||
dump_statistics( total, arena_counters_total );
|
||||
#if !__TBB_STATISTICS_TOTALS_ONLY
|
||||
printf( "==============================================\n" );
|
||||
#endif
|
||||
#endif /* __TBB_STATISTICS_STDOUT */
|
||||
}
|
||||
#endif /* __TBB_STATISTICS */
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
// The method inspects a scheduler to determine:
|
||||
// 1. if it has tasks that can be retrieved and executed (via the return value);
|
||||
// 2. if it has any tasks at all, including those of lower priority (via tasks_present);
|
||||
// 3. if it is able to work with enqueued tasks (via dequeuing_possible).
|
||||
inline bool arena::may_have_tasks ( generic_scheduler* s, bool& tasks_present, bool& dequeuing_possible ) {
|
||||
if ( !s
|
||||
#if __TBB_TASK_ARENA
|
||||
|| s->my_arena != this
|
||||
#endif
|
||||
) return false;
|
||||
dequeuing_possible |= s->worker_outermost_level();
|
||||
if ( s->my_pool_reshuffling_pending ) {
|
||||
// This primary task pool is nonempty and may contain tasks at the current
|
||||
// priority level. Its owner is winnowing lower priority tasks at the moment.
|
||||
tasks_present = true;
|
||||
return true;
|
||||
}
|
||||
if ( s->my_offloaded_tasks ) {
|
||||
tasks_present = true;
|
||||
if ( s->my_local_reload_epoch < *s->my_ref_reload_epoch ) {
|
||||
// This scheduler's offload area is nonempty and may contain tasks at the
|
||||
// current priority level.
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
void arena::orphan_offloaded_tasks(generic_scheduler& s) {
|
||||
__TBB_ASSERT( s.my_offloaded_tasks, NULL );
|
||||
GATHER_STATISTIC( ++s.my_counters.prio_orphanings );
|
||||
++my_abandonment_epoch;
|
||||
__TBB_ASSERT( s.my_offloaded_task_list_tail_link && !*s.my_offloaded_task_list_tail_link, NULL );
|
||||
task* orphans;
|
||||
do {
|
||||
orphans = const_cast<task*>(my_orphaned_tasks);
|
||||
*s.my_offloaded_task_list_tail_link = orphans;
|
||||
} while ( as_atomic(my_orphaned_tasks).compare_and_swap(s.my_offloaded_tasks, orphans) != orphans );
|
||||
s.my_offloaded_tasks = NULL;
|
||||
#if TBB_USE_ASSERT
|
||||
s.my_offloaded_task_list_tail_link = NULL;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
|
||||
bool arena::is_out_of_work() {
|
||||
// TODO: rework it to return at least a hint about where a task was found; better if the task itself.
|
||||
for(;;) {
|
||||
pool_state_t snapshot = my_pool_state;
|
||||
switch( snapshot ) {
|
||||
case SNAPSHOT_EMPTY:
|
||||
return true;
|
||||
case SNAPSHOT_FULL: {
|
||||
// Use unique id for "busy" in order to avoid ABA problems.
|
||||
const pool_state_t busy = pool_state_t(&busy);
|
||||
// Request permission to take snapshot
|
||||
if( my_pool_state.compare_and_swap( busy, SNAPSHOT_FULL )==SNAPSHOT_FULL ) {
|
||||
// Got permission. Take the snapshot.
|
||||
// NOTE: This is not a lock, as the state can be set to FULL at
|
||||
// any moment by a thread that spawns/enqueues new task.
|
||||
size_t n = my_limit;
|
||||
// Make local copies of volatile parameters. Their change during
|
||||
// snapshot taking procedure invalidates the attempt, and returns
|
||||
// this thread into the dispatch loop.
|
||||
#if __TBB_TASK_PRIORITY
|
||||
intptr_t top_priority = my_top_priority;
|
||||
uintptr_t reload_epoch = my_reload_epoch;
|
||||
// Inspect primary task pools first
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
size_t k;
|
||||
for( k=0; k<n; ++k ) {
|
||||
if( my_slots[k].task_pool != EmptyTaskPool &&
|
||||
__TBB_load_relaxed(my_slots[k].head) < __TBB_load_relaxed(my_slots[k].tail) )
|
||||
{
|
||||
// k-th primary task pool is nonempty and does contain tasks.
|
||||
break;
|
||||
}
|
||||
if( my_pool_state!=busy )
|
||||
return false; // the work was published
|
||||
}
|
||||
__TBB_ASSERT( k <= n, NULL );
|
||||
bool work_absent = k == n;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
// Variable tasks_present indicates presence of tasks at any priority
|
||||
// level, while work_absent refers only to the current priority.
|
||||
bool tasks_present = !work_absent || my_orphaned_tasks;
|
||||
bool dequeuing_possible = false;
|
||||
if ( work_absent ) {
|
||||
// Check for the possibility that recent priority changes
|
||||
// brought some tasks to the current priority level
|
||||
|
||||
uintptr_t abandonment_epoch = my_abandonment_epoch;
|
||||
// Master thread's scheduler needs special handling as it
|
||||
// may be destroyed at any moment (workers' schedulers are
|
||||
// guaranteed to be alive while at least one thread is in arena).
|
||||
// Have to exclude concurrency with task group state change propagation too.
|
||||
// TODO: check whether it is still necessary since some pools belong to slots now
|
||||
my_market->my_arenas_list_mutex.lock();
|
||||
generic_scheduler *s = my_slots[0].my_scheduler;
|
||||
if ( s && as_atomic(my_slots[0].my_scheduler).compare_and_swap(LockedMaster, s) == s ) { //TODO: remove need to lock
|
||||
__TBB_ASSERT( my_slots[0].my_scheduler == LockedMaster && s != LockedMaster, NULL );
|
||||
work_absent = !may_have_tasks( s, tasks_present, dequeuing_possible );
|
||||
__TBB_store_with_release( my_slots[0].my_scheduler, s );
|
||||
}
|
||||
my_market->my_arenas_list_mutex.unlock();
|
||||
// The following loop is subject to data races. While k-th slot's
|
||||
// scheduler is being examined, corresponding worker can either
|
||||
// leave to RML or migrate to another arena.
|
||||
// But the races are not prevented because all of them are benign.
|
||||
// First, the code relies on the fact that worker thread's scheduler
|
||||
// object persists until the whole library is deinitialized.
|
||||
// Second, in the worst case the races can only cause another
|
||||
// round of stealing attempts to be undertaken. Introducing complex
|
||||
// synchronization into this coldest part of the scheduler's control
|
||||
// flow does not seem to make sense because it both is unlikely to
|
||||
// ever have any observable performance effect, and will require
|
||||
// additional synchronization code on the hotter paths.
|
||||
for( k = 1; work_absent && k < n; ++k ) {
|
||||
if( my_pool_state!=busy )
|
||||
return false; // the work was published
|
||||
work_absent = !may_have_tasks( my_slots[k].my_scheduler, tasks_present, dequeuing_possible );
|
||||
}
|
||||
// Preclude premature switching arena off because of a race in the previous loop.
|
||||
work_absent = work_absent
|
||||
&& !__TBB_load_with_acquire(my_orphaned_tasks)
|
||||
&& abandonment_epoch == my_abandonment_epoch;
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
// Test and test-and-set.
|
||||
if( my_pool_state==busy ) {
|
||||
#if __TBB_TASK_PRIORITY
|
||||
bool no_fifo_tasks = my_task_stream[top_priority].empty();
|
||||
work_absent = work_absent && (!dequeuing_possible || no_fifo_tasks)
|
||||
&& top_priority == my_top_priority && reload_epoch == my_reload_epoch;
|
||||
#else
|
||||
bool no_fifo_tasks = my_task_stream.empty();
|
||||
work_absent = work_absent && no_fifo_tasks;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
if( work_absent ) {
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( top_priority > my_bottom_priority ) {
|
||||
if ( my_market->lower_arena_priority(*this, top_priority - 1, reload_epoch)
|
||||
&& !my_task_stream[top_priority].empty() )
|
||||
{
|
||||
atomic_update( my_skipped_fifo_priority, top_priority, std::less<intptr_t>());
|
||||
}
|
||||
}
|
||||
else if ( !tasks_present && !my_orphaned_tasks && no_fifo_tasks ) {
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
// save current demand value before setting SNAPSHOT_EMPTY,
|
||||
// to avoid race with advertise_new_work.
|
||||
int current_demand = (int)my_max_num_workers;
|
||||
if( my_pool_state.compare_and_swap( SNAPSHOT_EMPTY, busy )==busy ) {
|
||||
// This thread transitioned pool to empty state, and thus is
|
||||
// responsible for telling RML that there is no other work to do.
|
||||
my_market->adjust_demand( *this, -current_demand );
|
||||
#if __TBB_TASK_PRIORITY
|
||||
// Check for the presence of enqueued tasks "lost" on some of
|
||||
// priority levels because updating arena priority and switching
|
||||
// arena into "populated" (FULL) state happen non-atomically.
|
||||
// Imposing atomicity would require task::enqueue() to use a lock,
|
||||
// which is unacceptable.
|
||||
bool switch_back = false;
|
||||
for ( int p = 0; p < num_priority_levels; ++p ) {
|
||||
if ( !my_task_stream[p].empty() ) {
|
||||
switch_back = true;
|
||||
if ( p < my_bottom_priority || p > my_top_priority )
|
||||
my_market->update_arena_priority(*this, p);
|
||||
}
|
||||
}
|
||||
if ( switch_back )
|
||||
advertise_new_work</*Spawned*/false>();
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
}
|
||||
// Undo previous transition SNAPSHOT_FULL-->busy, unless another thread undid it.
|
||||
my_pool_state.compare_and_swap( SNAPSHOT_FULL, busy );
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
default:
|
||||
// Another thread is taking a snapshot.
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#if __TBB_COUNT_TASK_NODES
|
||||
intptr_t arena::workers_task_node_count() {
|
||||
intptr_t result = 0;
|
||||
for( unsigned i = 1; i < my_num_slots; ++i ) {
|
||||
generic_scheduler* s = my_slots[i].my_scheduler;
|
||||
if( s )
|
||||
result += s->my_task_node_count;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
#endif /* __TBB_COUNT_TASK_NODES */
|
||||
|
||||
void arena::enqueue_task( task& t, intptr_t prio, FastRandom &random )
|
||||
{
|
||||
#if __TBB_RECYCLE_TO_ENQUEUE
|
||||
__TBB_ASSERT( t.state()==task::allocated || t.state()==task::to_enqueue, "attempt to enqueue task with inappropriate state" );
|
||||
#else
|
||||
__TBB_ASSERT( t.state()==task::allocated, "attempt to enqueue task that is not in 'allocated' state" );
|
||||
#endif
|
||||
t.prefix().state = task::ready;
|
||||
t.prefix().extra_state |= es_task_enqueued; // enqueued task marker
|
||||
|
||||
#if TBB_USE_ASSERT
|
||||
if( task* parent = t.parent() ) {
|
||||
internal::reference_count ref_count = parent->prefix().ref_count;
|
||||
__TBB_ASSERT( ref_count!=0, "attempt to enqueue task whose parent has a ref_count==0 (forgot to set_ref_count?)" );
|
||||
__TBB_ASSERT( ref_count>0, "attempt to enqueue task whose parent has a ref_count<0" );
|
||||
parent->prefix().extra_state |= es_ref_count_active;
|
||||
}
|
||||
__TBB_ASSERT(t.prefix().affinity==affinity_id(0), "affinity is ignored for enqueued tasks");
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
intptr_t p = prio ? normalize_priority(priority_t(prio)) : normalized_normal_priority;
|
||||
assert_priority_valid(p);
|
||||
task_stream &ts = my_task_stream[p];
|
||||
#else /* !__TBB_TASK_PRIORITY */
|
||||
__TBB_ASSERT_EX(prio == 0, "the library is not configured to respect the task priority");
|
||||
task_stream &ts = my_task_stream;
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
ITT_NOTIFY(sync_releasing, &ts);
|
||||
ts.push( &t, random );
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( p != my_top_priority )
|
||||
my_market->update_arena_priority( *this, p );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
advertise_new_work< /*Spawned=*/ false >();
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( p != my_top_priority )
|
||||
my_market->update_arena_priority( *this, p );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
}
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
struct nested_arena_context : no_copy {
|
||||
generic_scheduler &my_scheduler;
|
||||
scheduler_state const my_orig_state;
|
||||
void *my_orig_ptr;
|
||||
bool my_adjusting;
|
||||
nested_arena_context(generic_scheduler *s, arena* a, bool needs_adjusting, bool as_worker = false)
|
||||
: my_scheduler(*s), my_orig_state(*s), my_orig_ptr(NULL), my_adjusting(needs_adjusting) {
|
||||
s->nested_arena_entry(a, *this, as_worker);
|
||||
}
|
||||
~nested_arena_context() {
|
||||
my_scheduler.nested_arena_exit(*this);
|
||||
(scheduler_state&)my_scheduler = my_orig_state; // restore arena settings
|
||||
}
|
||||
};
|
||||
|
||||
void generic_scheduler::nested_arena_entry(arena* a, nested_arena_context& c, bool as_worker) {
|
||||
if( a == my_arena ) {
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
c.my_orig_ptr = my_innermost_running_task =
|
||||
new(&allocate_task(sizeof(empty_task), NULL, a->my_default_ctx)) empty_task;
|
||||
#endif
|
||||
return;
|
||||
}
|
||||
__TBB_ASSERT( is_alive(a->my_guard), NULL );
|
||||
// overwrite arena settings
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( my_offloaded_tasks )
|
||||
my_arena->orphan_offloaded_tasks( *this );
|
||||
my_ref_top_priority = &a->my_top_priority;
|
||||
my_ref_reload_epoch = &a->my_reload_epoch;
|
||||
my_local_reload_epoch = a->my_reload_epoch;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
my_arena = a;
|
||||
my_arena_index = 0;
|
||||
my_arena_slot = my_arena->my_slots + my_arena_index;
|
||||
my_inbox.detach(); // TODO: mailboxes were not designed for switching, add copy constructor?
|
||||
attach_mailbox( affinity_id(my_arena_index+1) );
|
||||
my_innermost_running_task = my_dispatching_task = as_worker? NULL : my_dummy_task;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
// save dummy's context and replace it by arena's context
|
||||
c.my_orig_ptr = my_dummy_task->prefix().context;
|
||||
my_dummy_task->prefix().context = a->my_default_ctx;
|
||||
#endif
|
||||
#if __TBB_ARENA_OBSERVER
|
||||
my_last_local_observer = 0; // TODO: try optimize number of calls
|
||||
my_arena->my_observers.notify_entry_observers( my_last_local_observer, /*worker=*/false );
|
||||
#endif
|
||||
// TODO? ITT_NOTIFY(sync_acquired, a->my_slots + index);
|
||||
// TODO: it requires market to have P workers (not P-1)
|
||||
// TODO: it still allows temporary oversubscription by 1 worker (due to my_max_num_workers)
|
||||
// TODO: a preempted worker should be excluded from assignment to other arenas e.g. my_slack--
|
||||
if( c.my_adjusting ) my_arena->my_market->adjust_demand(*my_arena, -1);
|
||||
}
|
||||
|
||||
void generic_scheduler::nested_arena_exit(nested_arena_context& c) {
|
||||
if( my_arena == c.my_orig_state.my_arena ) {
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
free_task<small_local_task>(*(task*)c.my_orig_ptr); // TODO: use scoped_task instead?
|
||||
#endif
|
||||
return;
|
||||
}
|
||||
if( c.my_adjusting ) my_arena->my_market->adjust_demand(*my_arena, 1);
|
||||
#if __TBB_ARENA_OBSERVER
|
||||
my_arena->my_observers.notify_exit_observers( my_last_local_observer, /*worker=*/false );
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( my_offloaded_tasks )
|
||||
my_arena->orphan_offloaded_tasks( *this );
|
||||
my_local_reload_epoch = *c.my_orig_state.my_ref_reload_epoch;
|
||||
while ( as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap( NULL, this) != this )
|
||||
__TBB_Yield(); // TODO: task priority can use master slot for locking while accessing the scheduler
|
||||
#else
|
||||
// Free the master slot. TODO: support multiple masters
|
||||
__TBB_store_with_release(my_arena->my_slots[0].my_scheduler, (generic_scheduler*)NULL);
|
||||
#endif
|
||||
my_arena->my_exit_monitors.notify_all_relaxed(); // TODO: fix concurrent monitor to use notify_one (test MultipleMastersPart4 fails)
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
// restore context of dummy task
|
||||
my_dummy_task->prefix().context = (task_group_context*)c.my_orig_ptr;
|
||||
#endif
|
||||
}
|
||||
|
||||
void generic_scheduler::wait_until_empty() {
|
||||
my_dummy_task->prefix().ref_count++; // prevents exit from local_wait_for_all when local work is done enforcing the stealing
|
||||
while( my_arena->my_pool_state != arena::SNAPSHOT_EMPTY )
|
||||
local_wait_for_all(*my_dummy_task, NULL);
|
||||
my_dummy_task->prefix().ref_count--;
|
||||
}
|
||||
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
#include "scheduler_utility.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace interface7 {
|
||||
namespace internal {
|
||||
|
||||
void task_arena_base::internal_initialize( ) {
|
||||
__TBB_ASSERT( my_master_slots <= 1, "Number of slots reserved for master can be only [0,1]");
|
||||
if( my_master_slots > 1 ) my_master_slots = 1; // TODO: make more masters
|
||||
if( my_max_concurrency < 1 )
|
||||
my_max_concurrency = (int)governor::default_num_threads();
|
||||
// TODO: reimplement in an efficient way. We need a scheduler instance in this thread
|
||||
// but the scheduler is only required for task allocation and fifo random seeds until
|
||||
// master wants to join the arena. (Idea - to create a restricted specialization)
|
||||
// It is excessive to create an implicit arena for master here anyway. But scheduler
|
||||
// instance implies master thread to be always connected with arena.
|
||||
// browse recursively into init_scheduler and arena::process for details
|
||||
if( !governor::local_scheduler_if_initialized() )
|
||||
governor::init_scheduler( (unsigned)my_max_concurrency - my_master_slots + 1/*TODO: address in market instead*/, 0, true );
|
||||
// TODO: we will need to introduce a mechanism for global settings, including stack size, used by all arenas
|
||||
arena* new_arena = &market::create_arena( my_max_concurrency - my_master_slots/*it's +1 slot for num_masters=0*/, ThreadStackSize );
|
||||
if(as_atomic(my_arena).compare_and_swap(new_arena, NULL) != NULL) { // there is a race possible on my_initialized
|
||||
__TBB_ASSERT(my_arena, NULL); // other thread was the first
|
||||
new_arena->on_thread_leaving</*is_master*/true>(); // deallocate new arena
|
||||
}
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
else {
|
||||
my_context = new_arena->my_default_ctx;
|
||||
my_context->my_version_and_traits |= my_version_and_traits & exact_exception_flag;
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
void task_arena_base::internal_terminate( ) {
|
||||
if( my_arena ) {// task_arena was initialized
|
||||
#if __TBB_STATISTICS_EARLY_DUMP
|
||||
GATHER_STATISTIC( my_arena->dump_arena_statistics() );
|
||||
#endif
|
||||
my_arena->on_thread_leaving</*is_master*/true>();
|
||||
my_arena = 0;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
my_context = 0;
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
void task_arena_base::internal_enqueue( task& t, intptr_t prio ) const {
|
||||
__TBB_ASSERT(my_arena, NULL);
|
||||
generic_scheduler* s = governor::local_scheduler_if_initialized();
|
||||
__TBB_ASSERT(s, "Scheduler is not initialized"); // we allocated a task so can expect the scheduler
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_ASSERT(my_arena->my_default_ctx == t.prefix().context, NULL);
|
||||
__TBB_ASSERT(!my_arena->my_default_ctx->is_group_execution_cancelled(), // TODO: any better idea?
|
||||
"The task will not be executed because default task_group_context of task_arena is cancelled. Has previously enqueued task thrown an exception?");
|
||||
#endif
|
||||
my_arena->enqueue_task( t, prio, s->my_random );
|
||||
}
|
||||
|
||||
class delegated_task : public task {
|
||||
internal::delegate_base & my_delegate;
|
||||
concurrent_monitor & my_monitor;
|
||||
task * my_root;
|
||||
/*override*/ task* execute() {
|
||||
generic_scheduler& s = *(generic_scheduler*)prefix().owner;
|
||||
__TBB_ASSERT(s.worker_outermost_level() || s.master_outermost_level(), "expected to be enqueued and received on the outermost level");
|
||||
// but this task can mimics outermost level, detect it
|
||||
if( s.master_outermost_level() && s.my_dummy_task->state() == task::executing ) {
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
// RTTI is available, check whether the cast is valid
|
||||
__TBB_ASSERT(dynamic_cast<delegated_task*>(s.my_dummy_task), 0);
|
||||
#endif
|
||||
set_ref_count(1); // required by the semantics of recycle_to_enqueue()
|
||||
recycle_to_enqueue();
|
||||
return NULL;
|
||||
}
|
||||
struct outermost_context : internal::no_copy {
|
||||
delegated_task * t;
|
||||
generic_scheduler & s;
|
||||
task * orig_dummy;
|
||||
task_group_context * orig_ctx;
|
||||
outermost_context(delegated_task *_t, generic_scheduler &_s) : t(_t), s(_s) {
|
||||
orig_dummy = s.my_dummy_task;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
orig_ctx = t->prefix().context;
|
||||
t->prefix().context = s.my_arena->my_default_ctx;
|
||||
#endif
|
||||
s.my_dummy_task = t; // mimics outermost master
|
||||
__TBB_ASSERT(s.my_innermost_running_task == t, NULL);
|
||||
}
|
||||
~outermost_context() {
|
||||
s.my_dummy_task = orig_dummy;
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
// restore context for sake of registering potential exception
|
||||
t->prefix().context = orig_ctx;
|
||||
#endif
|
||||
}
|
||||
} scope(this, s);
|
||||
my_delegate();
|
||||
return NULL;
|
||||
}
|
||||
~delegated_task() {
|
||||
// potential exception was already registered. It must happen before the notification
|
||||
__TBB_ASSERT(my_root->ref_count()==2, NULL);
|
||||
__TBB_store_with_release(my_root->prefix().ref_count, 1); // must precede the wakeup
|
||||
my_monitor.notify_relaxed(*this);
|
||||
}
|
||||
public:
|
||||
delegated_task( internal::delegate_base & d, concurrent_monitor & s, task * t )
|
||||
: my_delegate(d), my_monitor(s), my_root(t) {}
|
||||
// predicate for concurrent_monitor notification
|
||||
bool operator()(uintptr_t ctx) const { return (void*)ctx == (void*)&my_delegate; }
|
||||
};
|
||||
|
||||
void task_arena_base::internal_execute( internal::delegate_base& d) const {
|
||||
__TBB_ASSERT(my_arena, NULL);
|
||||
generic_scheduler* s = governor::local_scheduler();
|
||||
__TBB_ASSERT(s, "Scheduler is not initialized");
|
||||
// TODO: is it safe to assign slot to a scheduler which is not yet switched?
|
||||
// TODO TEMP: one master, make more masters
|
||||
if( s->my_arena == my_arena || (!__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler)
|
||||
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL ) == NULL) ) {
|
||||
cpu_ctl_env_helper cpu_ctl_helper;
|
||||
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(my_context) );
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
try {
|
||||
#endif
|
||||
//TODO: replace dummy tasks for workers as well to avoid using of the_dummy_context
|
||||
nested_arena_context scope(s, my_arena, !my_master_slots);
|
||||
d();
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
} catch(...) {
|
||||
cpu_ctl_helper.restore_default(); // TODO: is it needed on Windows?
|
||||
if( my_version_and_traits & exact_exception_flag ) throw;
|
||||
else {
|
||||
task_group_context exception_container( task_group_context::isolated,
|
||||
task_group_context::default_traits & ~task_group_context::exact_exception );
|
||||
exception_container.register_pending_exception();
|
||||
__TBB_ASSERT(exception_container.my_exception, NULL);
|
||||
exception_container.my_exception->throw_self();
|
||||
}
|
||||
}
|
||||
#endif
|
||||
} else {
|
||||
concurrent_monitor::thread_context waiter;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
task_group_context exec_context( task_group_context::isolated, my_version_and_traits & exact_exception_flag );
|
||||
#if __TBB_FP_CONTEXT
|
||||
exec_context.copy_fp_settings( *my_context );
|
||||
#endif
|
||||
#endif
|
||||
auto_empty_task root(__TBB_CONTEXT_ARG(s, &exec_context));
|
||||
root.prefix().ref_count = 2;
|
||||
my_arena->enqueue_task( *new( task::allocate_root(__TBB_CONTEXT_ARG1(exec_context)) )
|
||||
delegated_task(d, my_arena->my_exit_monitors, &root),
|
||||
0, s->my_random ); // TODO: priority?
|
||||
do {
|
||||
my_arena->my_exit_monitors.prepare_wait(waiter, (uintptr_t)&d);
|
||||
if( __TBB_load_with_acquire(root.prefix().ref_count) < 2 ) {
|
||||
my_arena->my_exit_monitors.cancel_wait(waiter);
|
||||
break;
|
||||
}
|
||||
else if( !__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler) // TODO: refactor into a function?
|
||||
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL ) == NULL ) {
|
||||
my_arena->my_exit_monitors.cancel_wait(waiter);
|
||||
nested_arena_context scope(s, my_arena, !my_master_slots);
|
||||
s->local_wait_for_all(root, NULL);
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
__TBB_ASSERT( !exec_context.my_exception, NULL ); // exception can be thrown above, not deferred
|
||||
#endif
|
||||
__TBB_ASSERT( root.prefix().ref_count == 0, NULL );
|
||||
break;
|
||||
} else {
|
||||
my_arena->my_exit_monitors.commit_wait(waiter);
|
||||
}
|
||||
} while( __TBB_load_with_acquire(root.prefix().ref_count) == 2 );
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
// process possible exception
|
||||
if( task_group_context::exception_container_type *pe = exec_context.my_exception )
|
||||
pe->throw_self();
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
// this wait task is a temporary approach to wait for arena emptiness for masters without slots
|
||||
// TODO: it will be rather reworked for one source of notification from is_out_of_work
|
||||
class wait_task : public task {
|
||||
binary_semaphore & my_signal;
|
||||
/*override*/ task* execute() {
|
||||
generic_scheduler* s = governor::local_scheduler_if_initialized();
|
||||
__TBB_ASSERT( s, NULL );
|
||||
if( s->my_arena_index && s->worker_outermost_level() ) {// on outermost level of workers only
|
||||
s->local_wait_for_all( *s->my_dummy_task, NULL ); // run remaining tasks
|
||||
} else s->my_arena->is_out_of_work(); // avoids starvation of internal_wait: issuing this task makes arena full
|
||||
my_signal.V();
|
||||
return NULL;
|
||||
}
|
||||
public:
|
||||
wait_task ( binary_semaphore & sema ) : my_signal(sema) {}
|
||||
};
|
||||
|
||||
void task_arena_base::internal_wait() const {
|
||||
__TBB_ASSERT(my_arena, NULL);
|
||||
generic_scheduler* s = governor::local_scheduler();
|
||||
__TBB_ASSERT(s, "Scheduler is not initialized");
|
||||
__TBB_ASSERT(s->my_arena != my_arena || s->my_arena_index == 0, "task_arena::wait_until_empty() is not supported within a worker context" );
|
||||
if( s->my_arena == my_arena ) {
|
||||
//unsupported, but try do something for outermost master
|
||||
__TBB_ASSERT(s->master_outermost_level(), "unsupported");
|
||||
if( !s->my_arena_index )
|
||||
while( my_arena->num_workers_active() )
|
||||
s->wait_until_empty();
|
||||
} else for(;;) {
|
||||
while( my_arena->my_pool_state != arena::SNAPSHOT_EMPTY ) {
|
||||
if( !__TBB_load_with_acquire(my_arena->my_slots[0].my_scheduler) // TODO TEMP: one master, make more masters
|
||||
&& as_atomic(my_arena->my_slots[0].my_scheduler).compare_and_swap(s, NULL) == NULL ) {
|
||||
nested_arena_context a(s, my_arena, !my_master_slots, true);
|
||||
s->wait_until_empty();
|
||||
} else {
|
||||
binary_semaphore waiter; // TODO: replace by a single event notification from is_out_of_work
|
||||
internal_enqueue( *new( task::allocate_root(__TBB_CONTEXT_ARG1(*my_context)) ) wait_task(waiter), 0 ); // TODO: priority?
|
||||
waiter.P(); // TODO: concurrent_monitor
|
||||
}
|
||||
}
|
||||
if( !my_arena->num_workers_active() && !my_arena->my_slots[0].my_scheduler) // no activity
|
||||
break; // spin until workers active but avoid spinning in a worker
|
||||
__TBB_Yield(); // wait until workers and master leave
|
||||
}
|
||||
}
|
||||
|
||||
/*static*/ int task_arena_base::internal_current_slot() {
|
||||
generic_scheduler* s = governor::local_scheduler_if_initialized();
|
||||
return s? int(s->my_arena_index) : -1;
|
||||
}
|
||||
|
||||
|
||||
} // tbb::interfaceX::internal
|
||||
} // tbb::interfaceX
|
||||
} // tbb
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
396
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/arena.h
vendored
Normal file
396
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/arena.h
vendored
Normal file
@@ -0,0 +1,396 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_arena_H
|
||||
#define _TBB_arena_H
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
#include "tbb/atomic.h"
|
||||
|
||||
#include "tbb/tbb_machine.h"
|
||||
|
||||
#include "scheduler_common.h"
|
||||
#include "intrusive_list.h"
|
||||
#include "task_stream.h"
|
||||
#include "../rml/include/rml_tbb.h"
|
||||
#include "mailbox.h"
|
||||
#include "observer_proxy.h"
|
||||
#include "market.h"
|
||||
#include "governor.h"
|
||||
#if __TBB_TASK_ARENA
|
||||
#include "concurrent_monitor.h"
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
|
||||
class task_group_context;
|
||||
class allocate_root_with_context_proxy;
|
||||
|
||||
namespace internal {
|
||||
|
||||
//! arena data except the array of slots
|
||||
/** Separated in order to simplify padding.
|
||||
Intrusive list node base class is used by market to form a list of arenas. **/
|
||||
struct arena_base : padded<intrusive_list_node> {
|
||||
//! Number of workers that have been marked out by the resource manager to service the arena
|
||||
unsigned my_num_workers_allotted; // heavy use in stealing loop
|
||||
|
||||
//! References of the arena
|
||||
/** Counts workers and master references separately. Bit 0 indicates reference from implicit
|
||||
master or explicit task_arena; the next bits contain number of workers servicing the arena.*/
|
||||
atomic<unsigned> my_references; // heavy use in stealing loop
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
//! Highest priority of recently spawned or enqueued tasks.
|
||||
volatile intptr_t my_top_priority; // heavy use in stealing loop
|
||||
|
||||
//! Maximal currently busy slot.
|
||||
atomic<unsigned> my_limit; // heavy use in stealing loop
|
||||
|
||||
//! Task pool for the tasks scheduled via task::enqueue() method
|
||||
/** Such scheduling guarantees eventual execution even if
|
||||
- new tasks are constantly coming (by extracting scheduled tasks in
|
||||
relaxed FIFO order);
|
||||
- the enqueuing thread does not call any of wait_for_all methods. **/
|
||||
task_stream my_task_stream[num_priority_levels]; // heavy use in stealing loop
|
||||
#else /* !__TBB_TASK_PRIORITY */
|
||||
//! Task pool for the tasks scheduled via task::enqueue() method
|
||||
/** Such scheduling guarantees eventual execution even if
|
||||
- new tasks are constantly coming (by extracting scheduled tasks in
|
||||
relaxed FIFO order);
|
||||
- the enqueuing thread does not call any of wait_for_all methods. **/
|
||||
task_stream my_task_stream; // heavy use in stealing loop
|
||||
|
||||
//! Maximal currently busy slot.
|
||||
atomic<unsigned> my_limit; // heavy use in stealing loop
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
|
||||
//! Number of workers that are currently requested from the resource manager
|
||||
int my_num_workers_requested;
|
||||
|
||||
//! Number of slots in the arena
|
||||
unsigned my_num_slots;
|
||||
|
||||
//! Number of workers requested by the master thread owning the arena
|
||||
unsigned my_max_num_workers;
|
||||
|
||||
//! Market owning this arena
|
||||
market* my_market;
|
||||
|
||||
//! ABA prevention marker
|
||||
uintptr_t my_aba_epoch;
|
||||
|
||||
#if !__TBB_FP_CONTEXT
|
||||
//! FPU control settings of arena's master thread captured at the moment of arena instantiation.
|
||||
__TBB_cpu_ctl_env_t my_cpu_ctl_env;
|
||||
#endif
|
||||
|
||||
#if __TBB_TRACK_PRIORITY_LEVEL_SATURATION
|
||||
int my_num_workers_present;
|
||||
#endif /* __TBB_TRACK_PRIORITY_LEVEL_SATURATION */
|
||||
|
||||
//! Current task pool state and estimate of available tasks amount.
|
||||
/** The estimate is either 0 (SNAPSHOT_EMPTY) or infinity (SNAPSHOT_FULL).
|
||||
Special state is "busy" (any other unsigned value).
|
||||
Note that the implementation of arena::is_busy_or_empty() requires
|
||||
my_pool_state to be unsigned. */
|
||||
tbb::atomic<uintptr_t> my_pool_state;
|
||||
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
//! Default task group context.
|
||||
/** Used by root tasks allocated directly by the master thread (not from inside
|
||||
a TBB task) without explicit context specification. **/
|
||||
task_group_context* my_default_ctx;
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
//! List of local observers attached to this arena.
|
||||
observer_list my_observers;
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
//! Lowest normalized priority of available spawned or enqueued tasks.
|
||||
intptr_t my_bottom_priority;
|
||||
|
||||
//! Tracks events that may bring tasks in offload areas to the top priority level.
|
||||
/** Incremented when arena top priority changes or a task group priority
|
||||
is elevated to the current arena's top level. **/
|
||||
uintptr_t my_reload_epoch;
|
||||
|
||||
//! List of offloaded tasks abandoned by workers revoked by the market
|
||||
task* my_orphaned_tasks;
|
||||
|
||||
//! Counter used to track the occurrence of recent orphaning and re-sharing operations.
|
||||
tbb::atomic<uintptr_t> my_abandonment_epoch;
|
||||
|
||||
//! Highest priority level containing enqueued tasks
|
||||
/** It being greater than 0 means that high priority enqueued tasks had to be
|
||||
bypassed because all workers were blocked in nested dispatch loops and
|
||||
were unable to progress at then current priority level. **/
|
||||
tbb::atomic<intptr_t> my_skipped_fifo_priority;
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
|
||||
//! Indicates if there is an oversubscribing worker created to service enqueued tasks.
|
||||
bool my_mandatory_concurrency;
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
//! exit notifications after arena slot is released
|
||||
concurrent_monitor my_exit_monitors;
|
||||
#endif
|
||||
|
||||
#if TBB_USE_ASSERT
|
||||
//! Used to trap accesses to the object after its destruction.
|
||||
uintptr_t my_guard;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
}; // struct arena_base
|
||||
|
||||
class arena
|
||||
#if (__GNUC__<4 || __GNUC__==4 && __GNUC_MINOR__==0) && !__INTEL_COMPILER
|
||||
: public padded<arena_base>
|
||||
#else
|
||||
: private padded<arena_base>
|
||||
#endif
|
||||
{
|
||||
private:
|
||||
friend class generic_scheduler;
|
||||
template<typename SchedulerTraits> friend class custom_scheduler;
|
||||
friend class governor;
|
||||
friend class task_scheduler_observer_v3;
|
||||
friend class market;
|
||||
friend class tbb::task;
|
||||
friend class tbb::task_group_context;
|
||||
friend class allocate_root_with_context_proxy;
|
||||
friend class intrusive_list<arena>;
|
||||
friend class interface7::internal::task_arena_base; // declared in scheduler_common.h
|
||||
friend class interface7::internal::delegated_task;
|
||||
friend class interface7::internal::wait_task;
|
||||
|
||||
typedef padded<arena_base> base_type;
|
||||
|
||||
//! Constructor
|
||||
arena ( market&, unsigned max_num_workers );
|
||||
|
||||
//! Allocate an instance of arena.
|
||||
static arena& allocate_arena( market&, unsigned max_num_workers );
|
||||
|
||||
static int unsigned num_slots_to_reserve ( unsigned max_num_workers ) {
|
||||
return max(2u, max_num_workers + 1);
|
||||
}
|
||||
|
||||
static int allocation_size ( unsigned max_num_workers ) {
|
||||
return sizeof(base_type) + num_slots_to_reserve(max_num_workers) * (sizeof(mail_outbox) + sizeof(arena_slot));
|
||||
}
|
||||
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
//! Finds all contexts affected by the state change and propagates the new state to them.
|
||||
/** The propagation is relayed to the market because tasks created by one
|
||||
master thread can be passed to and executed by other masters. This means
|
||||
that context trees can span several arenas at once and thus state change
|
||||
propagation cannot be generally localized to one arena only. **/
|
||||
template <typename T>
|
||||
bool propagate_task_group_state ( T task_group_context::*mptr_state, task_group_context& src, T new_state );
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
//! Get reference to mailbox corresponding to given affinity_id.
|
||||
mail_outbox& mailbox( affinity_id id ) {
|
||||
__TBB_ASSERT( 0<id, "affinity id must be positive integer" );
|
||||
__TBB_ASSERT( id <= my_num_slots, "affinity id out of bounds" );
|
||||
|
||||
return ((mail_outbox*)this)[-(int)id];
|
||||
}
|
||||
|
||||
//! Completes arena shutdown, destructs and deallocates it.
|
||||
void free_arena ();
|
||||
|
||||
typedef uintptr_t pool_state_t;
|
||||
|
||||
//! No tasks to steal since last snapshot was taken
|
||||
static const pool_state_t SNAPSHOT_EMPTY = 0;
|
||||
|
||||
//! At least one task has been offered for stealing since the last snapshot started
|
||||
static const pool_state_t SNAPSHOT_FULL = pool_state_t(-1);
|
||||
|
||||
//! No tasks to steal or snapshot is being taken.
|
||||
static bool is_busy_or_empty( pool_state_t s ) { return s < SNAPSHOT_FULL; }
|
||||
|
||||
//! The number of workers active in the arena.
|
||||
unsigned num_workers_active( ) {
|
||||
return my_references >> 1;
|
||||
}
|
||||
|
||||
//! If necessary, raise a flag that there is new job in arena.
|
||||
template<bool Spawned> void advertise_new_work();
|
||||
|
||||
//! Check if there is job anywhere in arena.
|
||||
/** Return true if no job or if arena is being cleaned up. */
|
||||
bool is_out_of_work();
|
||||
|
||||
//! enqueue a task into starvation-resistance queue
|
||||
void enqueue_task( task&, intptr_t, FastRandom & );
|
||||
|
||||
//! Registers the worker with the arena and enters TBB scheduler dispatch loop
|
||||
void process( generic_scheduler& );
|
||||
|
||||
//! Notification that worker or master leaves its arena
|
||||
template<bool is_master>
|
||||
inline void on_thread_leaving ( );
|
||||
|
||||
#if __TBB_STATISTICS
|
||||
//! Outputs internal statistics accumulated by the arena
|
||||
void dump_arena_statistics ();
|
||||
#endif /* __TBB_STATISTICS */
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
//! Check if recent priority changes may bring some tasks to the current priority level soon
|
||||
/** /param tasks_present indicates presence of tasks at any priority level. **/
|
||||
inline bool may_have_tasks ( generic_scheduler*, bool& tasks_present, bool& dequeuing_possible );
|
||||
|
||||
//! Puts offloaded tasks into global list of orphaned tasks
|
||||
void orphan_offloaded_tasks ( generic_scheduler& s );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
|
||||
#if __TBB_COUNT_TASK_NODES
|
||||
//! Returns the number of task objects "living" in worker threads
|
||||
intptr_t workers_task_node_count();
|
||||
#endif
|
||||
|
||||
/** Must be the last data field */
|
||||
arena_slot my_slots[1];
|
||||
}; // class arena
|
||||
|
||||
|
||||
template<bool is_master>
|
||||
inline void arena::on_thread_leaving ( ) {
|
||||
//
|
||||
// Implementation of arena destruction synchronization logic contained various
|
||||
// bugs/flaws at the different stages of its evolution, so below is a detailed
|
||||
// description of the issues taken into consideration in the framework of the
|
||||
// current design.
|
||||
//
|
||||
// In case of using fire-and-forget tasks (scheduled via task::enqueue())
|
||||
// master thread is allowed to leave its arena before all its work is executed,
|
||||
// and market may temporarily revoke all workers from this arena. Since revoked
|
||||
// workers never attempt to reset arena state to EMPTY and cancel its request
|
||||
// to RML for threads, the arena object is destroyed only when both the last
|
||||
// thread is leaving it and arena's state is EMPTY (that is its master thread
|
||||
// left and it does not contain any work).
|
||||
//
|
||||
// A worker that checks for work presence and transitions arena to the EMPTY
|
||||
// state (in snapshot taking procedure arena::is_out_of_work()) updates
|
||||
// arena::my_pool_state first and only then arena::my_num_workers_requested.
|
||||
// So the check for work absence must be done against the latter field.
|
||||
//
|
||||
// In a time window between decrementing the active threads count and checking
|
||||
// if there is an outstanding request for workers. New worker thread may arrive,
|
||||
// finish remaining work, set arena state to empty, and leave decrementing its
|
||||
// refcount and destroying. Then the current thread will destroy the arena
|
||||
// the second time. To preclude it a local copy of the outstanding request
|
||||
// value can be stored before decrementing active threads count.
|
||||
//
|
||||
// But this technique may cause two other problem. When the stored request is
|
||||
// zero, it is possible that arena still has threads and they can generate new
|
||||
// tasks and thus re-establish non-zero requests. Then all the threads can be
|
||||
// revoked (as described above) leaving this thread the last one, and causing
|
||||
// it to destroy non-empty arena.
|
||||
//
|
||||
// The other problem takes place when the stored request is non-zero. Another
|
||||
// thread may complete the work, set arena state to empty, and leave without
|
||||
// arena destruction before this thread decrements the refcount. This thread
|
||||
// cannot destroy the arena either. Thus the arena may be "orphaned".
|
||||
//
|
||||
// In both cases we cannot dereference arena pointer after the refcount is
|
||||
// decremented, as our arena may already be destroyed.
|
||||
//
|
||||
// If this is the master thread, market can be concurrently destroyed.
|
||||
// In case of workers market's liveness is ensured by the RML connection
|
||||
// rundown protocol, according to which the client (i.e. the market) lives
|
||||
// until RML server notifies it about connection termination, and this
|
||||
// notification is fired only after all workers return into RML.
|
||||
//
|
||||
// Thus if we decremented refcount to zero we ask the market to check arena
|
||||
// state (including the fact if it is alive) under the lock.
|
||||
//
|
||||
uintptr_t aba_epoch = my_aba_epoch;
|
||||
market* m = my_market;
|
||||
__TBB_ASSERT(my_references > int(!is_master), "broken arena reference counter");
|
||||
if ( (my_references -= is_master? 1:2 ) == 0 ) // worker's counter starts from bit 1
|
||||
market::try_destroy_arena( m, this, aba_epoch, is_master );
|
||||
}
|
||||
|
||||
template<bool Spawned> void arena::advertise_new_work() {
|
||||
if( !Spawned ) { // i.e. the work was enqueued
|
||||
if( my_max_num_workers==0 ) {
|
||||
my_max_num_workers = 1;
|
||||
__TBB_ASSERT(!my_mandatory_concurrency, "");
|
||||
my_mandatory_concurrency = true;
|
||||
__TBB_ASSERT(!num_workers_active(), "");
|
||||
my_pool_state = SNAPSHOT_FULL;
|
||||
my_market->adjust_demand( *this, 1 );
|
||||
return;
|
||||
}
|
||||
// Local memory fence is required to avoid missed wakeups; see the comment below.
|
||||
// Starvation resistant tasks require mandatory concurrency, so missed wakeups are unacceptable.
|
||||
atomic_fence();
|
||||
}
|
||||
// Double-check idiom that, in case of spawning, is deliberately sloppy about memory fences.
|
||||
// Technically, to avoid missed wakeups, there should be a full memory fence between the point we
|
||||
// released the task pool (i.e. spawned task) and read the arena's state. However, adding such a
|
||||
// fence might hurt overall performance more than it helps, because the fence would be executed
|
||||
// on every task pool release, even when stealing does not occur. Since TBB allows parallelism,
|
||||
// but never promises parallelism, the missed wakeup is not a correctness problem.
|
||||
pool_state_t snapshot = my_pool_state;
|
||||
if( is_busy_or_empty(snapshot) ) {
|
||||
// Attempt to mark as full. The compare_and_swap below is a little unusual because the
|
||||
// result is compared to a value that can be different than the comparand argument.
|
||||
if( my_pool_state.compare_and_swap( SNAPSHOT_FULL, snapshot )==SNAPSHOT_EMPTY ) {
|
||||
if( snapshot!=SNAPSHOT_EMPTY ) {
|
||||
// This thread read "busy" into snapshot, and then another thread transitioned
|
||||
// my_pool_state to "empty" in the meantime, which caused the compare_and_swap above
|
||||
// to fail. Attempt to transition my_pool_state from "empty" to "full".
|
||||
if( my_pool_state.compare_and_swap( SNAPSHOT_FULL, SNAPSHOT_EMPTY )!=SNAPSHOT_EMPTY ) {
|
||||
// Some other thread transitioned my_pool_state from "empty", and hence became
|
||||
// responsible for waking up workers.
|
||||
return;
|
||||
}
|
||||
}
|
||||
// This thread transitioned pool from empty to full state, and thus is responsible for
|
||||
// telling RML that there is work to do.
|
||||
if( Spawned ) {
|
||||
if( my_mandatory_concurrency ) {
|
||||
__TBB_ASSERT(my_max_num_workers==1, "");
|
||||
__TBB_ASSERT(!governor::local_scheduler()->is_worker(), "");
|
||||
// There was deliberate oversubscription on 1 core for sake of starvation-resistant tasks.
|
||||
// Now a single active thread (must be the master) supposedly starts a new parallel region
|
||||
// with relaxed sequential semantics, and oversubscription should be avoided.
|
||||
// Demand for workers has been decreased to 0 during SNAPSHOT_EMPTY, so just keep it.
|
||||
my_max_num_workers = 0;
|
||||
my_mandatory_concurrency = false;
|
||||
return;
|
||||
}
|
||||
}
|
||||
my_market->adjust_demand( *this, my_max_num_workers );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* _TBB_arena_H */
|
||||
556
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/atomic.h
vendored
Normal file
556
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/atomic.h
vendored
Normal file
@@ -0,0 +1,556 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_atomic_H
|
||||
#define __TBB_atomic_H
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
#if _MSC_VER
|
||||
#define __TBB_LONG_LONG __int64
|
||||
#else
|
||||
#define __TBB_LONG_LONG long long
|
||||
#endif /* _MSC_VER */
|
||||
|
||||
#include "tbb_machine.h"
|
||||
|
||||
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
// Workaround for overzealous compiler warnings
|
||||
#pragma warning (push)
|
||||
#pragma warning (disable: 4244 4267 4512)
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
|
||||
//! Specifies memory semantics.
|
||||
enum memory_semantics {
|
||||
//! Sequential consistency
|
||||
full_fence,
|
||||
//! Acquire
|
||||
acquire,
|
||||
//! Release
|
||||
release,
|
||||
//! No ordering
|
||||
relaxed
|
||||
};
|
||||
|
||||
//! @cond INTERNAL
|
||||
namespace internal {
|
||||
|
||||
#if __TBB_ATTRIBUTE_ALIGNED_PRESENT
|
||||
#define __TBB_DECL_ATOMIC_FIELD(t,f,a) t f __attribute__ ((aligned(a)));
|
||||
#elif __TBB_DECLSPEC_ALIGN_PRESENT
|
||||
#define __TBB_DECL_ATOMIC_FIELD(t,f,a) __declspec(align(a)) t f;
|
||||
#else
|
||||
#error Do not know syntax for forcing alignment.
|
||||
#endif
|
||||
|
||||
template<size_t S>
|
||||
struct atomic_rep; // Primary template declared, but never defined.
|
||||
|
||||
template<>
|
||||
struct atomic_rep<1> { // Specialization
|
||||
typedef int8_t word;
|
||||
};
|
||||
template<>
|
||||
struct atomic_rep<2> { // Specialization
|
||||
typedef int16_t word;
|
||||
};
|
||||
template<>
|
||||
struct atomic_rep<4> { // Specialization
|
||||
#if _MSC_VER && !_WIN64
|
||||
// Work-around that avoids spurious /Wp64 warnings
|
||||
typedef intptr_t word;
|
||||
#else
|
||||
typedef int32_t word;
|
||||
#endif
|
||||
};
|
||||
#if __TBB_64BIT_ATOMICS
|
||||
template<>
|
||||
struct atomic_rep<8> { // Specialization
|
||||
typedef int64_t word;
|
||||
};
|
||||
#endif
|
||||
|
||||
template<typename value_type, size_t size>
|
||||
struct aligned_storage;
|
||||
|
||||
//the specializations are needed to please MSVC syntax of __declspec(align()) which accept _literal_ constants only
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
#define ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(S) \
|
||||
template<typename value_type> \
|
||||
struct aligned_storage<value_type,S> { \
|
||||
__TBB_DECL_ATOMIC_FIELD(value_type,my_value,S) \
|
||||
aligned_storage() = default ; \
|
||||
constexpr aligned_storage(value_type value):my_value(value){} \
|
||||
}; \
|
||||
|
||||
#else
|
||||
#define ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(S) \
|
||||
template<typename value_type> \
|
||||
struct aligned_storage<value_type,S> { \
|
||||
__TBB_DECL_ATOMIC_FIELD(value_type,my_value,S) \
|
||||
}; \
|
||||
|
||||
#endif
|
||||
|
||||
template<typename value_type>
|
||||
struct aligned_storage<value_type,1> {
|
||||
value_type my_value;
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
aligned_storage() = default ;
|
||||
constexpr aligned_storage(value_type value):my_value(value){}
|
||||
#endif
|
||||
};
|
||||
|
||||
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(2)
|
||||
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(4)
|
||||
#if __TBB_64BIT_ATOMICS
|
||||
ATOMIC_STORAGE_PARTIAL_SPECIALIZATION(8)
|
||||
#endif
|
||||
|
||||
template<size_t Size, memory_semantics M>
|
||||
struct atomic_traits; // Primary template declared, but not defined.
|
||||
|
||||
#define __TBB_DECL_FENCED_ATOMIC_PRIMITIVES(S,M) \
|
||||
template<> struct atomic_traits<S,M> { \
|
||||
typedef atomic_rep<S>::word word; \
|
||||
inline static word compare_and_swap( volatile void* location, word new_value, word comparand ) { \
|
||||
return __TBB_machine_cmpswp##S##M(location,new_value,comparand); \
|
||||
} \
|
||||
inline static word fetch_and_add( volatile void* location, word addend ) { \
|
||||
return __TBB_machine_fetchadd##S##M(location,addend); \
|
||||
} \
|
||||
inline static word fetch_and_store( volatile void* location, word value ) { \
|
||||
return __TBB_machine_fetchstore##S##M(location,value); \
|
||||
} \
|
||||
};
|
||||
|
||||
#define __TBB_DECL_ATOMIC_PRIMITIVES(S) \
|
||||
template<memory_semantics M> \
|
||||
struct atomic_traits<S,M> { \
|
||||
typedef atomic_rep<S>::word word; \
|
||||
inline static word compare_and_swap( volatile void* location, word new_value, word comparand ) { \
|
||||
return __TBB_machine_cmpswp##S(location,new_value,comparand); \
|
||||
} \
|
||||
inline static word fetch_and_add( volatile void* location, word addend ) { \
|
||||
return __TBB_machine_fetchadd##S(location,addend); \
|
||||
} \
|
||||
inline static word fetch_and_store( volatile void* location, word value ) { \
|
||||
return __TBB_machine_fetchstore##S(location,value); \
|
||||
} \
|
||||
};
|
||||
|
||||
template<memory_semantics M>
|
||||
struct atomic_load_store_traits; // Primary template declaration
|
||||
|
||||
#define __TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(M) \
|
||||
template<> struct atomic_load_store_traits<M> { \
|
||||
template <typename T> \
|
||||
inline static T load( const volatile T& location ) { \
|
||||
return __TBB_load_##M( location ); \
|
||||
} \
|
||||
template <typename T> \
|
||||
inline static void store( volatile T& location, T value ) { \
|
||||
__TBB_store_##M( location, value ); \
|
||||
} \
|
||||
}
|
||||
|
||||
#if __TBB_USE_FENCED_ATOMICS
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,full_fence)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,full_fence)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,full_fence)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,acquire)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,acquire)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,acquire)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,release)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,release)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,release)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(1,relaxed)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(2,relaxed)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(4,relaxed)
|
||||
#if __TBB_64BIT_ATOMICS
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,full_fence)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,acquire)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,release)
|
||||
__TBB_DECL_FENCED_ATOMIC_PRIMITIVES(8,relaxed)
|
||||
#endif
|
||||
#else /* !__TBB_USE_FENCED_ATOMICS */
|
||||
__TBB_DECL_ATOMIC_PRIMITIVES(1)
|
||||
__TBB_DECL_ATOMIC_PRIMITIVES(2)
|
||||
__TBB_DECL_ATOMIC_PRIMITIVES(4)
|
||||
#if __TBB_64BIT_ATOMICS
|
||||
__TBB_DECL_ATOMIC_PRIMITIVES(8)
|
||||
#endif
|
||||
#endif /* !__TBB_USE_FENCED_ATOMICS */
|
||||
|
||||
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(full_fence);
|
||||
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(acquire);
|
||||
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(release);
|
||||
__TBB_DECL_ATOMIC_LOAD_STORE_PRIMITIVES(relaxed);
|
||||
|
||||
//! Additive inverse of 1 for type T.
|
||||
/** Various compilers issue various warnings if -1 is used with various integer types.
|
||||
The baroque expression below avoids all the warnings (we hope). */
|
||||
#define __TBB_MINUS_ONE(T) (T(T(0)-T(1)))
|
||||
|
||||
//! Base class that provides basic functionality for atomic<T> without fetch_and_add.
|
||||
/** Works for any type T that has the same size as an integral type, has a trivial constructor/destructor,
|
||||
and can be copied/compared by memcpy/memcmp. */
|
||||
template<typename T>
|
||||
struct atomic_impl {
|
||||
protected:
|
||||
aligned_storage<T,sizeof(T)> my_storage;
|
||||
private:
|
||||
//TODO: rechecks on recent versions of gcc if union is still the _only_ way to do a conversion without warnings
|
||||
//! Union type used to convert type T to underlying integral type.
|
||||
template<typename value_type>
|
||||
union converter {
|
||||
typedef typename atomic_rep<sizeof(value_type)>::word bits_type;
|
||||
converter(){}
|
||||
converter(value_type a_value) : value(a_value) {}
|
||||
value_type value;
|
||||
bits_type bits;
|
||||
};
|
||||
|
||||
template<typename value_t>
|
||||
static typename converter<value_t>::bits_type to_bits(value_t value){
|
||||
return converter<value_t>(value).bits;
|
||||
}
|
||||
template<typename value_t>
|
||||
static value_t to_value(typename converter<value_t>::bits_type bits){
|
||||
converter<value_t> u;
|
||||
u.bits = bits;
|
||||
return u.value;
|
||||
}
|
||||
|
||||
template<typename value_t>
|
||||
union ptr_converter; //Primary template declared, but never defined.
|
||||
|
||||
template<typename value_t>
|
||||
union ptr_converter<value_t *> {
|
||||
ptr_converter(){}
|
||||
ptr_converter(value_t* a_value) : value(a_value) {}
|
||||
value_t* value;
|
||||
uintptr_t bits;
|
||||
};
|
||||
//TODO: check if making to_bits accepting reference (thus unifying it with to_bits_ref)
|
||||
//does not hurt performance
|
||||
template<typename value_t>
|
||||
static typename converter<value_t>::bits_type & to_bits_ref(value_t& value){
|
||||
//TODO: this #ifdef is temporary workaround, as union conversion seems to fail
|
||||
//on suncc for 64 bit types for 32 bit target
|
||||
#if !__SUNPRO_CC
|
||||
return *(typename converter<value_t>::bits_type*)ptr_converter<value_t*>(&value).bits;
|
||||
#else
|
||||
return *(typename converter<value_t>::bits_type*)(&value);
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
public:
|
||||
typedef T value_type;
|
||||
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
atomic_impl() = default ;
|
||||
constexpr atomic_impl(value_type value):my_storage(value){}
|
||||
#endif
|
||||
template<memory_semantics M>
|
||||
value_type fetch_and_store( value_type value ) {
|
||||
return to_value<value_type>(
|
||||
internal::atomic_traits<sizeof(value_type),M>::fetch_and_store( &my_storage.my_value, to_bits(value) )
|
||||
);
|
||||
}
|
||||
|
||||
value_type fetch_and_store( value_type value ) {
|
||||
return fetch_and_store<full_fence>(value);
|
||||
}
|
||||
|
||||
template<memory_semantics M>
|
||||
value_type compare_and_swap( value_type value, value_type comparand ) {
|
||||
return to_value<value_type>(
|
||||
internal::atomic_traits<sizeof(value_type),M>::compare_and_swap( &my_storage.my_value, to_bits(value), to_bits(comparand) )
|
||||
);
|
||||
}
|
||||
|
||||
value_type compare_and_swap( value_type value, value_type comparand ) {
|
||||
return compare_and_swap<full_fence>(value,comparand);
|
||||
}
|
||||
|
||||
operator value_type() const volatile { // volatile qualifier here for backwards compatibility
|
||||
return to_value<value_type>(
|
||||
__TBB_load_with_acquire( to_bits_ref(my_storage.my_value) )
|
||||
);
|
||||
}
|
||||
|
||||
template<memory_semantics M>
|
||||
value_type load () const {
|
||||
return to_value<value_type>(
|
||||
internal::atomic_load_store_traits<M>::load( to_bits_ref(my_storage.my_value) )
|
||||
);
|
||||
}
|
||||
|
||||
value_type load () const {
|
||||
return load<acquire>();
|
||||
}
|
||||
|
||||
template<memory_semantics M>
|
||||
void store ( value_type value ) {
|
||||
internal::atomic_load_store_traits<M>::store( to_bits_ref(my_storage.my_value), to_bits(value));
|
||||
}
|
||||
|
||||
void store ( value_type value ) {
|
||||
store<release>( value );
|
||||
}
|
||||
|
||||
protected:
|
||||
value_type store_with_release( value_type rhs ) {
|
||||
//TODO: unify with store<release>
|
||||
__TBB_store_with_release( to_bits_ref(my_storage.my_value), to_bits(rhs) );
|
||||
return rhs;
|
||||
}
|
||||
};
|
||||
|
||||
//! Base class that provides basic functionality for atomic<T> with fetch_and_add.
|
||||
/** I is the underlying type.
|
||||
D is the difference type.
|
||||
StepType should be char if I is an integral type, and T if I is a T*. */
|
||||
template<typename I, typename D, typename StepType>
|
||||
struct atomic_impl_with_arithmetic: atomic_impl<I> {
|
||||
public:
|
||||
typedef I value_type;
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
atomic_impl_with_arithmetic() = default ;
|
||||
constexpr atomic_impl_with_arithmetic(value_type value): atomic_impl<I>(value){}
|
||||
#endif
|
||||
template<memory_semantics M>
|
||||
value_type fetch_and_add( D addend ) {
|
||||
return value_type(internal::atomic_traits<sizeof(value_type),M>::fetch_and_add( &this->my_storage.my_value, addend*sizeof(StepType) ));
|
||||
}
|
||||
|
||||
value_type fetch_and_add( D addend ) {
|
||||
return fetch_and_add<full_fence>(addend);
|
||||
}
|
||||
|
||||
template<memory_semantics M>
|
||||
value_type fetch_and_increment() {
|
||||
return fetch_and_add<M>(1);
|
||||
}
|
||||
|
||||
value_type fetch_and_increment() {
|
||||
return fetch_and_add(1);
|
||||
}
|
||||
|
||||
template<memory_semantics M>
|
||||
value_type fetch_and_decrement() {
|
||||
return fetch_and_add<M>(__TBB_MINUS_ONE(D));
|
||||
}
|
||||
|
||||
value_type fetch_and_decrement() {
|
||||
return fetch_and_add(__TBB_MINUS_ONE(D));
|
||||
}
|
||||
|
||||
public:
|
||||
value_type operator+=( D value ) {
|
||||
return fetch_and_add(value)+value;
|
||||
}
|
||||
|
||||
value_type operator-=( D value ) {
|
||||
// Additive inverse of value computed using binary minus,
|
||||
// instead of unary minus, for sake of avoiding compiler warnings.
|
||||
return operator+=(D(0)-value);
|
||||
}
|
||||
|
||||
value_type operator++() {
|
||||
return fetch_and_add(1)+1;
|
||||
}
|
||||
|
||||
value_type operator--() {
|
||||
return fetch_and_add(__TBB_MINUS_ONE(D))-1;
|
||||
}
|
||||
|
||||
value_type operator++(int) {
|
||||
return fetch_and_add(1);
|
||||
}
|
||||
|
||||
value_type operator--(int) {
|
||||
return fetch_and_add(__TBB_MINUS_ONE(D));
|
||||
}
|
||||
};
|
||||
|
||||
} /* Internal */
|
||||
//! @endcond
|
||||
|
||||
//! Primary template for atomic.
|
||||
/** See the Reference for details.
|
||||
@ingroup synchronization */
|
||||
template<typename T>
|
||||
struct atomic: internal::atomic_impl<T> {
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
atomic() = default;
|
||||
constexpr atomic(T arg): internal::atomic_impl<T>(arg) {}
|
||||
#endif
|
||||
T operator=( T rhs ) {
|
||||
// "this" required here in strict ISO C++ because store_with_release is a dependent name
|
||||
return this->store_with_release(rhs);
|
||||
}
|
||||
atomic<T>& operator=( const atomic<T>& rhs ) {this->store_with_release(rhs); return *this;}
|
||||
};
|
||||
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
#define __TBB_DECL_ATOMIC(T) \
|
||||
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
|
||||
atomic() = default; \
|
||||
constexpr atomic(T arg): internal::atomic_impl_with_arithmetic<T,T,char>(arg) {} \
|
||||
\
|
||||
T operator=( T rhs ) {return store_with_release(rhs);} \
|
||||
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
|
||||
};
|
||||
#else
|
||||
#define __TBB_DECL_ATOMIC(T) \
|
||||
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
|
||||
T operator=( T rhs ) {return store_with_release(rhs);} \
|
||||
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
|
||||
};
|
||||
#endif
|
||||
|
||||
#if __TBB_64BIT_ATOMICS
|
||||
//TODO: consider adding non-default (and atomic) copy constructor for 32bit platform
|
||||
__TBB_DECL_ATOMIC(__TBB_LONG_LONG)
|
||||
__TBB_DECL_ATOMIC(unsigned __TBB_LONG_LONG)
|
||||
#else
|
||||
// test_atomic will verify that sizeof(long long)==8
|
||||
#endif
|
||||
__TBB_DECL_ATOMIC(long)
|
||||
__TBB_DECL_ATOMIC(unsigned long)
|
||||
|
||||
#if _MSC_VER && !_WIN64
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
/* Special version of __TBB_DECL_ATOMIC that avoids gratuitous warnings from cl /Wp64 option.
|
||||
It is identical to __TBB_DECL_ATOMIC(unsigned) except that it replaces operator=(T)
|
||||
with an operator=(U) that explicitly converts the U to a T. Types T and U should be
|
||||
type synonyms on the platform. Type U should be the wider variant of T from the
|
||||
perspective of /Wp64. */
|
||||
#define __TBB_DECL_ATOMIC_ALT(T,U) \
|
||||
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
|
||||
atomic() = default ; \
|
||||
constexpr atomic(T arg): internal::atomic_impl_with_arithmetic<T,T,char>(arg) {} \
|
||||
T operator=( U rhs ) {return store_with_release(T(rhs));} \
|
||||
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
|
||||
};
|
||||
#else
|
||||
#define __TBB_DECL_ATOMIC_ALT(T,U) \
|
||||
template<> struct atomic<T>: internal::atomic_impl_with_arithmetic<T,T,char> { \
|
||||
T operator=( U rhs ) {return store_with_release(T(rhs));} \
|
||||
atomic<T>& operator=( const atomic<T>& rhs ) {store_with_release(rhs); return *this;} \
|
||||
};
|
||||
#endif
|
||||
__TBB_DECL_ATOMIC_ALT(unsigned,size_t)
|
||||
__TBB_DECL_ATOMIC_ALT(int,ptrdiff_t)
|
||||
#else
|
||||
__TBB_DECL_ATOMIC(unsigned)
|
||||
__TBB_DECL_ATOMIC(int)
|
||||
#endif /* _MSC_VER && !_WIN64 */
|
||||
|
||||
__TBB_DECL_ATOMIC(unsigned short)
|
||||
__TBB_DECL_ATOMIC(short)
|
||||
__TBB_DECL_ATOMIC(char)
|
||||
__TBB_DECL_ATOMIC(signed char)
|
||||
__TBB_DECL_ATOMIC(unsigned char)
|
||||
|
||||
#if !_MSC_VER || defined(_NATIVE_WCHAR_T_DEFINED)
|
||||
__TBB_DECL_ATOMIC(wchar_t)
|
||||
#endif /* _MSC_VER||!defined(_NATIVE_WCHAR_T_DEFINED) */
|
||||
|
||||
//! Specialization for atomic<T*> with arithmetic and operator->.
|
||||
template<typename T> struct atomic<T*>: internal::atomic_impl_with_arithmetic<T*,ptrdiff_t,T> {
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
atomic() = default ;
|
||||
constexpr atomic(T* arg): internal::atomic_impl_with_arithmetic<T*,ptrdiff_t,T>(arg) {}
|
||||
#endif
|
||||
T* operator=( T* rhs ) {
|
||||
// "this" required here in strict ISO C++ because store_with_release is a dependent name
|
||||
return this->store_with_release(rhs);
|
||||
}
|
||||
atomic<T*>& operator=( const atomic<T*>& rhs ) {
|
||||
this->store_with_release(rhs); return *this;
|
||||
}
|
||||
T* operator->() const {
|
||||
return (*this);
|
||||
}
|
||||
};
|
||||
|
||||
//! Specialization for atomic<void*>, for sake of not allowing arithmetic or operator->.
|
||||
template<> struct atomic<void*>: internal::atomic_impl<void*> {
|
||||
#if __TBB_ATOMIC_CTORS
|
||||
atomic() = default ;
|
||||
constexpr atomic(void* arg): internal::atomic_impl<void*>(arg) {}
|
||||
#endif
|
||||
void* operator=( void* rhs ) {
|
||||
// "this" required here in strict ISO C++ because store_with_release is a dependent name
|
||||
return this->store_with_release(rhs);
|
||||
}
|
||||
atomic<void*>& operator=( const atomic<void*>& rhs ) {
|
||||
this->store_with_release(rhs); return *this;
|
||||
}
|
||||
};
|
||||
|
||||
// Helpers to workaround ugly syntax of calling template member function of a
|
||||
// template class with template argument dependent on template parameters.
|
||||
|
||||
template <memory_semantics M, typename T>
|
||||
T load ( const atomic<T>& a ) { return a.template load<M>(); }
|
||||
|
||||
template <memory_semantics M, typename T>
|
||||
void store ( atomic<T>& a, T value ) { a.template store<M>(value); }
|
||||
|
||||
namespace interface6{
|
||||
//! Make an atomic for use in an initialization (list), as an alternative to zero-initialization or normal assignment.
|
||||
template<typename T>
|
||||
atomic<T> make_atomic(T t) {
|
||||
atomic<T> a;
|
||||
store<relaxed>(a,t);
|
||||
return a;
|
||||
}
|
||||
}
|
||||
using interface6::make_atomic;
|
||||
|
||||
namespace internal {
|
||||
template<memory_semantics M, typename T >
|
||||
void swap(atomic<T> & lhs, atomic<T> & rhs){
|
||||
T tmp = load<M>(lhs);
|
||||
store<M>(lhs,load<M>(rhs));
|
||||
store<M>(rhs,tmp);
|
||||
}
|
||||
|
||||
// only to aid in the gradual conversion of ordinary variables to proper atomics
|
||||
template<typename T>
|
||||
inline atomic<T>& as_atomic( T& t ) {
|
||||
return (atomic<T>&)t;
|
||||
}
|
||||
} // namespace tbb::internal
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#if _MSC_VER && !__INTEL_COMPILER
|
||||
#pragma warning (pop)
|
||||
#endif // warnings 4244, 4267 are back
|
||||
|
||||
#endif /* __TBB_atomic_H */
|
||||
@@ -0,0 +1,159 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_blocked_range_H
|
||||
#define __TBB_blocked_range_H
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
/** \page range_req Requirements on range concept
|
||||
Class \c R implementing the concept of range must define:
|
||||
- \code R::R( const R& ); \endcode Copy constructor
|
||||
- \code R::~R(); \endcode Destructor
|
||||
- \code bool R::is_divisible() const; \endcode True if range can be partitioned into two subranges
|
||||
- \code bool R::empty() const; \endcode True if range is empty
|
||||
- \code R::R( R& r, split ); \endcode Split range \c r into two subranges.
|
||||
**/
|
||||
|
||||
//! A range over which to iterate.
|
||||
/** @ingroup algorithms */
|
||||
template<typename Value>
|
||||
class blocked_range {
|
||||
public:
|
||||
//! Type of a value
|
||||
/** Called a const_iterator for sake of algorithms that need to treat a blocked_range
|
||||
as an STL container. */
|
||||
typedef Value const_iterator;
|
||||
|
||||
//! Type for size of a range
|
||||
typedef std::size_t size_type;
|
||||
|
||||
//! Construct range with default-constructed values for begin and end.
|
||||
/** Requires that Value have a default constructor. */
|
||||
blocked_range() : my_end(), my_begin() {}
|
||||
|
||||
//! Construct range over half-open interval [begin,end), with the given grainsize.
|
||||
blocked_range( Value begin_, Value end_, size_type grainsize_=1 ) :
|
||||
my_end(end_), my_begin(begin_), my_grainsize(grainsize_)
|
||||
{
|
||||
__TBB_ASSERT( my_grainsize>0, "grainsize must be positive" );
|
||||
}
|
||||
|
||||
//! Beginning of range.
|
||||
const_iterator begin() const {return my_begin;}
|
||||
|
||||
//! One past last value in range.
|
||||
const_iterator end() const {return my_end;}
|
||||
|
||||
//! Size of the range
|
||||
/** Unspecified if end()<begin(). */
|
||||
size_type size() const {
|
||||
__TBB_ASSERT( !(end()<begin()), "size() unspecified if end()<begin()" );
|
||||
return size_type(my_end-my_begin);
|
||||
}
|
||||
|
||||
//! The grain size for this range.
|
||||
size_type grainsize() const {return my_grainsize;}
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// Methods that implement Range concept
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
//! True if range is empty.
|
||||
bool empty() const {return !(my_begin<my_end);}
|
||||
|
||||
//! True if range is divisible.
|
||||
/** Unspecified if end()<begin(). */
|
||||
bool is_divisible() const {return my_grainsize<size();}
|
||||
|
||||
//! Split range.
|
||||
/** The new Range *this has the second part, the old range r has the first part.
|
||||
Unspecified if end()<begin() or !is_divisible(). */
|
||||
blocked_range( blocked_range& r, split ) :
|
||||
my_end(r.my_end),
|
||||
my_begin(do_split(r, split())),
|
||||
my_grainsize(r.my_grainsize)
|
||||
{
|
||||
// only comparison 'less than' is required from values of blocked_range objects
|
||||
__TBB_ASSERT( !(my_begin < r.my_end) && !(r.my_end < my_begin), "blocked_range has been split incorrectly" );
|
||||
}
|
||||
|
||||
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
|
||||
//! Static field to support proportional split
|
||||
static const bool is_divisible_in_proportion = true;
|
||||
|
||||
//! Split range.
|
||||
/** The new Range *this has the second part split according to specified proportion, the old range r has the first part.
|
||||
Unspecified if end()<begin() or !is_divisible(). */
|
||||
blocked_range( blocked_range& r, proportional_split& proportion ) :
|
||||
my_end(r.my_end),
|
||||
my_begin(do_split(r, proportion)),
|
||||
my_grainsize(r.my_grainsize)
|
||||
{
|
||||
// only comparison 'less than' is required from values of blocked_range objects
|
||||
__TBB_ASSERT( !(my_begin < r.my_end) && !(r.my_end < my_begin), "blocked_range has been split incorrectly" );
|
||||
}
|
||||
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
|
||||
|
||||
private:
|
||||
/** NOTE: my_end MUST be declared before my_begin, otherwise the forking constructor will break. */
|
||||
Value my_end;
|
||||
Value my_begin;
|
||||
size_type my_grainsize;
|
||||
|
||||
//! Auxiliary function used by forking constructor.
|
||||
/** Using this function lets us not require that Value support assignment or default construction. */
|
||||
static Value do_split( blocked_range& r, split )
|
||||
{
|
||||
__TBB_ASSERT( r.is_divisible(), "cannot split blocked_range that is not divisible" );
|
||||
Value middle = r.my_begin + (r.my_end - r.my_begin) / 2u;
|
||||
r.my_end = middle;
|
||||
return middle;
|
||||
}
|
||||
|
||||
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
|
||||
static Value do_split( blocked_range& r, proportional_split& proportion )
|
||||
{
|
||||
__TBB_ASSERT( r.is_divisible(), "cannot split blocked_range that is not divisible" );
|
||||
|
||||
// usage of 32-bit floating point arithmetic is not enough to handle ranges of
|
||||
// more than 2^24 iterations accurately. However, even on ranges with 2^64
|
||||
// iterations the computational error approximately equals to 0.000001% which
|
||||
// makes small impact on uniform distribution of such range's iterations (assuming
|
||||
// all iterations take equal time to complete). See 'test_partitioner_whitebox'
|
||||
// for implementation of an exact split algorithm
|
||||
size_type right_part = size_type(float(r.size()) * float(proportion.right())
|
||||
/ float(proportion.left() + proportion.right()) + 0.5f);
|
||||
return r.my_end = Value(r.my_end - right_part);
|
||||
}
|
||||
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
|
||||
|
||||
template<typename RowValue, typename ColValue>
|
||||
friend class blocked_range2d;
|
||||
|
||||
template<typename RowValue, typename ColValue, typename PageValue>
|
||||
friend class blocked_range3d;
|
||||
};
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_blocked_range_H */
|
||||
@@ -0,0 +1,108 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_blocked_range2d_H
|
||||
#define __TBB_blocked_range2d_H
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
#include "blocked_range.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
//! A 2-dimensional range that models the Range concept.
|
||||
/** @ingroup algorithms */
|
||||
template<typename RowValue, typename ColValue=RowValue>
|
||||
class blocked_range2d {
|
||||
public:
|
||||
//! Type for size of an iteration range
|
||||
typedef blocked_range<RowValue> row_range_type;
|
||||
typedef blocked_range<ColValue> col_range_type;
|
||||
|
||||
private:
|
||||
row_range_type my_rows;
|
||||
col_range_type my_cols;
|
||||
|
||||
public:
|
||||
|
||||
blocked_range2d( RowValue row_begin, RowValue row_end, typename row_range_type::size_type row_grainsize,
|
||||
ColValue col_begin, ColValue col_end, typename col_range_type::size_type col_grainsize ) :
|
||||
my_rows(row_begin,row_end,row_grainsize),
|
||||
my_cols(col_begin,col_end,col_grainsize)
|
||||
{
|
||||
}
|
||||
|
||||
blocked_range2d( RowValue row_begin, RowValue row_end,
|
||||
ColValue col_begin, ColValue col_end ) :
|
||||
my_rows(row_begin,row_end),
|
||||
my_cols(col_begin,col_end)
|
||||
{
|
||||
}
|
||||
|
||||
//! True if range is empty
|
||||
bool empty() const {
|
||||
// Yes, it is a logical OR here, not AND.
|
||||
return my_rows.empty() || my_cols.empty();
|
||||
}
|
||||
|
||||
//! True if range is divisible into two pieces.
|
||||
bool is_divisible() const {
|
||||
return my_rows.is_divisible() || my_cols.is_divisible();
|
||||
}
|
||||
|
||||
blocked_range2d( blocked_range2d& r, split ) :
|
||||
my_rows(r.my_rows),
|
||||
my_cols(r.my_cols)
|
||||
{
|
||||
split split_obj;
|
||||
do_split(r, split_obj);
|
||||
}
|
||||
|
||||
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
|
||||
//! Static field to support proportional split
|
||||
static const bool is_divisible_in_proportion = true;
|
||||
|
||||
blocked_range2d( blocked_range2d& r, proportional_split& proportion ) :
|
||||
my_rows(r.my_rows),
|
||||
my_cols(r.my_cols)
|
||||
{
|
||||
do_split(r, proportion);
|
||||
}
|
||||
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
|
||||
|
||||
template <typename Split>
|
||||
void do_split( blocked_range2d& r, Split& split_obj )
|
||||
{
|
||||
if( my_rows.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_rows.grainsize()) ) {
|
||||
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
|
||||
} else {
|
||||
my_rows.my_begin = row_range_type::do_split(r.my_rows, split_obj);
|
||||
}
|
||||
}
|
||||
|
||||
//! The rows of the iteration space
|
||||
const row_range_type& rows() const {return my_rows;}
|
||||
|
||||
//! The columns of the iteration space
|
||||
const col_range_type& cols() const {return my_cols;}
|
||||
};
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_blocked_range2d_H */
|
||||
@@ -0,0 +1,128 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_blocked_range3d_H
|
||||
#define __TBB_blocked_range3d_H
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
#include "blocked_range.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
//! A 3-dimensional range that models the Range concept.
|
||||
/** @ingroup algorithms */
|
||||
template<typename PageValue, typename RowValue=PageValue, typename ColValue=RowValue>
|
||||
class blocked_range3d {
|
||||
public:
|
||||
//! Type for size of an iteration range
|
||||
typedef blocked_range<PageValue> page_range_type;
|
||||
typedef blocked_range<RowValue> row_range_type;
|
||||
typedef blocked_range<ColValue> col_range_type;
|
||||
|
||||
private:
|
||||
page_range_type my_pages;
|
||||
row_range_type my_rows;
|
||||
col_range_type my_cols;
|
||||
|
||||
public:
|
||||
|
||||
blocked_range3d( PageValue page_begin, PageValue page_end,
|
||||
RowValue row_begin, RowValue row_end,
|
||||
ColValue col_begin, ColValue col_end ) :
|
||||
my_pages(page_begin,page_end),
|
||||
my_rows(row_begin,row_end),
|
||||
my_cols(col_begin,col_end)
|
||||
{
|
||||
}
|
||||
|
||||
blocked_range3d( PageValue page_begin, PageValue page_end, typename page_range_type::size_type page_grainsize,
|
||||
RowValue row_begin, RowValue row_end, typename row_range_type::size_type row_grainsize,
|
||||
ColValue col_begin, ColValue col_end, typename col_range_type::size_type col_grainsize ) :
|
||||
my_pages(page_begin,page_end,page_grainsize),
|
||||
my_rows(row_begin,row_end,row_grainsize),
|
||||
my_cols(col_begin,col_end,col_grainsize)
|
||||
{
|
||||
}
|
||||
|
||||
//! True if range is empty
|
||||
bool empty() const {
|
||||
// Yes, it is a logical OR here, not AND.
|
||||
return my_pages.empty() || my_rows.empty() || my_cols.empty();
|
||||
}
|
||||
|
||||
//! True if range is divisible into two pieces.
|
||||
bool is_divisible() const {
|
||||
return my_pages.is_divisible() || my_rows.is_divisible() || my_cols.is_divisible();
|
||||
}
|
||||
|
||||
blocked_range3d( blocked_range3d& r, split ) :
|
||||
my_pages(r.my_pages),
|
||||
my_rows(r.my_rows),
|
||||
my_cols(r.my_cols)
|
||||
{
|
||||
split split_obj;
|
||||
do_split(r, split_obj);
|
||||
}
|
||||
|
||||
#if __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES
|
||||
//! Static field to support proportional split
|
||||
static const bool is_divisible_in_proportion = true;
|
||||
|
||||
blocked_range3d( blocked_range3d& r, proportional_split& proportion ) :
|
||||
my_pages(r.my_pages),
|
||||
my_rows(r.my_rows),
|
||||
my_cols(r.my_cols)
|
||||
{
|
||||
do_split(r, proportion);
|
||||
}
|
||||
#endif /* __TBB_USE_PROPORTIONAL_SPLIT_IN_BLOCKED_RANGES */
|
||||
|
||||
template <typename Split>
|
||||
void do_split( blocked_range3d& r, Split& split_obj)
|
||||
{
|
||||
if ( my_pages.size()*double(my_rows.grainsize()) < my_rows.size()*double(my_pages.grainsize()) ) {
|
||||
if ( my_rows.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_rows.grainsize()) ) {
|
||||
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
|
||||
} else {
|
||||
my_rows.my_begin = row_range_type::do_split(r.my_rows, split_obj);
|
||||
}
|
||||
} else {
|
||||
if ( my_pages.size()*double(my_cols.grainsize()) < my_cols.size()*double(my_pages.grainsize()) ) {
|
||||
my_cols.my_begin = col_range_type::do_split(r.my_cols, split_obj);
|
||||
} else {
|
||||
my_pages.my_begin = page_range_type::do_split(r.my_pages, split_obj);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//! The pages of the iteration space
|
||||
const page_range_type& pages() const {return my_pages;}
|
||||
|
||||
//! The rows of the iteration space
|
||||
const row_range_type& rows() const {return my_rows;}
|
||||
|
||||
//! The columns of the iteration space
|
||||
const col_range_type& cols() const {return my_cols;}
|
||||
|
||||
};
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_blocked_range3d_H */
|
||||
@@ -0,0 +1,256 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
#include "tbb/cache_aligned_allocator.h"
|
||||
#include "tbb/tbb_allocator.h"
|
||||
#include "tbb/tbb_exception.h"
|
||||
#include "tbb_misc.h"
|
||||
#include "dynamic_link.h"
|
||||
#include <cstdlib>
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#include "tbb/machine/windows_api.h"
|
||||
#else
|
||||
#include <dlfcn.h>
|
||||
#endif /* _WIN32||_WIN64 */
|
||||
|
||||
using namespace std;
|
||||
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
|
||||
#pragma weak scalable_malloc
|
||||
#pragma weak scalable_free
|
||||
#pragma weak scalable_aligned_malloc
|
||||
#pragma weak scalable_aligned_free
|
||||
|
||||
extern "C" {
|
||||
void* scalable_malloc( size_t );
|
||||
void scalable_free( void* );
|
||||
void* scalable_aligned_malloc( size_t, size_t );
|
||||
void scalable_aligned_free( void* );
|
||||
}
|
||||
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
|
||||
//! Dummy routine used for first indirect call via MallocHandler.
|
||||
static void* DummyMalloc( size_t size );
|
||||
|
||||
//! Dummy routine used for first indirect call via FreeHandler.
|
||||
static void DummyFree( void * ptr );
|
||||
|
||||
//! Handler for memory allocation
|
||||
static void* (*MallocHandler)( size_t size ) = &DummyMalloc;
|
||||
|
||||
//! Handler for memory deallocation
|
||||
static void (*FreeHandler)( void* pointer ) = &DummyFree;
|
||||
|
||||
//! Dummy routine used for first indirect call via padded_allocate_handler.
|
||||
static void* dummy_padded_allocate( size_t bytes, size_t alignment );
|
||||
|
||||
//! Dummy routine used for first indirect call via padded_free_handler.
|
||||
static void dummy_padded_free( void * ptr );
|
||||
|
||||
// ! Allocates memory using standard malloc. It is used when scalable_allocator is not available
|
||||
static void* padded_allocate( size_t bytes, size_t alignment );
|
||||
|
||||
// ! Allocates memory using standard free. It is used when scalable_allocator is not available
|
||||
static void padded_free( void* p );
|
||||
|
||||
//! Handler for padded memory allocation
|
||||
static void* (*padded_allocate_handler)( size_t bytes, size_t alignment ) = &dummy_padded_allocate;
|
||||
|
||||
//! Handler for padded memory deallocation
|
||||
static void (*padded_free_handler)( void* p ) = &dummy_padded_free;
|
||||
|
||||
//! Table describing how to link the handlers.
|
||||
static const dynamic_link_descriptor MallocLinkTable[] = {
|
||||
DLD(scalable_malloc, MallocHandler),
|
||||
DLD(scalable_free, FreeHandler),
|
||||
DLD(scalable_aligned_malloc, padded_allocate_handler),
|
||||
DLD(scalable_aligned_free, padded_free_handler),
|
||||
};
|
||||
|
||||
|
||||
#if TBB_USE_DEBUG
|
||||
#define DEBUG_SUFFIX "_debug"
|
||||
#else
|
||||
#define DEBUG_SUFFIX
|
||||
#endif /* TBB_USE_DEBUG */
|
||||
|
||||
// MALLOCLIB_NAME is the name of the TBB memory allocator library.
|
||||
#if _WIN32||_WIN64
|
||||
#define MALLOCLIB_NAME "tbbmalloc" DEBUG_SUFFIX ".dll"
|
||||
#elif __APPLE__
|
||||
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX ".dylib"
|
||||
#elif __FreeBSD__ || __NetBSD__ || __sun || _AIX || __ANDROID__
|
||||
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX ".so"
|
||||
#elif __linux__ // Note that order of these #elif's is important!
|
||||
#define MALLOCLIB_NAME "libtbbmalloc" DEBUG_SUFFIX __TBB_STRING(.so.TBB_COMPATIBLE_INTERFACE_VERSION)
|
||||
#else
|
||||
#error Unknown OS
|
||||
#endif
|
||||
|
||||
//! Initialize the allocation/free handler pointers.
|
||||
/** Caller is responsible for ensuring this routine is called exactly once.
|
||||
The routine attempts to dynamically link with the TBB memory allocator.
|
||||
If that allocator is not found, it links to malloc and free. */
|
||||
void initialize_handler_pointers() {
|
||||
__TBB_ASSERT( MallocHandler==&DummyMalloc, NULL );
|
||||
bool success = dynamic_link( MALLOCLIB_NAME, MallocLinkTable, 4 );
|
||||
if( !success ) {
|
||||
// If unsuccessful, set the handlers to the default routines.
|
||||
// This must be done now, and not before FillDynamicLinks runs, because if other
|
||||
// threads call the handlers, we want them to go through the DoOneTimeInitializations logic,
|
||||
// which forces them to wait.
|
||||
FreeHandler = &free;
|
||||
MallocHandler = &malloc;
|
||||
padded_allocate_handler = &padded_allocate;
|
||||
padded_free_handler = &padded_free;
|
||||
}
|
||||
#if !__TBB_RML_STATIC
|
||||
PrintExtraVersionInfo( "ALLOCATOR", success?"scalable_malloc":"malloc" );
|
||||
#endif
|
||||
}
|
||||
|
||||
static tbb::atomic<do_once_state> initialization_state;
|
||||
void initialize_cache_aligned_allocator() {
|
||||
atomic_do_once( &initialize_handler_pointers, initialization_state );
|
||||
}
|
||||
|
||||
//! Executed on very first call through MallocHandler
|
||||
static void* DummyMalloc( size_t size ) {
|
||||
initialize_cache_aligned_allocator();
|
||||
__TBB_ASSERT( MallocHandler!=&DummyMalloc, NULL );
|
||||
return (*MallocHandler)( size );
|
||||
}
|
||||
|
||||
//! Executed on very first call through FreeHandler
|
||||
static void DummyFree( void * ptr ) {
|
||||
initialize_cache_aligned_allocator();
|
||||
__TBB_ASSERT( FreeHandler!=&DummyFree, NULL );
|
||||
(*FreeHandler)( ptr );
|
||||
}
|
||||
|
||||
//! Executed on very first call through padded_allocate_handler
|
||||
static void* dummy_padded_allocate( size_t bytes, size_t alignment ) {
|
||||
initialize_cache_aligned_allocator();
|
||||
__TBB_ASSERT( padded_allocate_handler!=&dummy_padded_allocate, NULL );
|
||||
return (*padded_allocate_handler)(bytes, alignment);
|
||||
}
|
||||
|
||||
//! Executed on very first call through padded_free_handler
|
||||
static void dummy_padded_free( void * ptr ) {
|
||||
initialize_cache_aligned_allocator();
|
||||
__TBB_ASSERT( padded_free_handler!=&dummy_padded_free, NULL );
|
||||
(*padded_free_handler)( ptr );
|
||||
}
|
||||
|
||||
static size_t NFS_LineSize = 128;
|
||||
|
||||
size_t NFS_GetLineSize() {
|
||||
return NFS_LineSize;
|
||||
}
|
||||
|
||||
#if _MSC_VER && !defined(__INTEL_COMPILER)
|
||||
// unary minus operator applied to unsigned type, result still unsigned
|
||||
#pragma warning( disable: 4146 4706 )
|
||||
#endif
|
||||
|
||||
void* NFS_Allocate( size_t n, size_t element_size, void* /*hint*/ ) {
|
||||
size_t m = NFS_LineSize;
|
||||
__TBB_ASSERT( m<=NFS_MaxLineSize, "illegal value for NFS_LineSize" );
|
||||
__TBB_ASSERT( (m & (m-1))==0, "must be power of two" );
|
||||
size_t bytes = n*element_size;
|
||||
|
||||
if (bytes<n || bytes+m<bytes) {
|
||||
// Overflow
|
||||
throw_exception(eid_bad_alloc);
|
||||
}
|
||||
// scalable_aligned_malloc considers zero size request an error, and returns NULL
|
||||
if (bytes==0) bytes = 1;
|
||||
|
||||
void* result = (*padded_allocate_handler)( bytes, m );
|
||||
if (!result)
|
||||
throw_exception(eid_bad_alloc);
|
||||
|
||||
__TBB_ASSERT( ((size_t)result&(m-1)) == 0, "The address returned isn't aligned to cache line size" );
|
||||
return result;
|
||||
}
|
||||
|
||||
void NFS_Free( void* p ) {
|
||||
(*padded_free_handler)( p );
|
||||
}
|
||||
|
||||
static void* padded_allocate( size_t bytes, size_t alignment ) {
|
||||
unsigned char* result = NULL;
|
||||
unsigned char* base = (unsigned char*)malloc(alignment+bytes);
|
||||
if( base ) {
|
||||
// Round up to the next line
|
||||
result = (unsigned char*)((uintptr_t)(base+alignment)&-alignment);
|
||||
// Record where block actually starts.
|
||||
((uintptr_t*)result)[-1] = uintptr_t(base);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
static void padded_free( void* p ) {
|
||||
if( p ) {
|
||||
__TBB_ASSERT( (uintptr_t)p>=0x4096, "attempt to free block not obtained from cache_aligned_allocator" );
|
||||
// Recover where block actually starts
|
||||
unsigned char* base = ((unsigned char**)p)[-1];
|
||||
__TBB_ASSERT( (void*)((uintptr_t)(base+NFS_LineSize)&-NFS_LineSize)==p, "not allocated by NFS_Allocate?" );
|
||||
free(base);
|
||||
}
|
||||
}
|
||||
|
||||
void* __TBB_EXPORTED_FUNC allocate_via_handler_v3( size_t n ) {
|
||||
void* result = (*MallocHandler) (n);
|
||||
if (!result) {
|
||||
throw_exception(eid_bad_alloc);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
void __TBB_EXPORTED_FUNC deallocate_via_handler_v3( void *p ) {
|
||||
if( p ) {
|
||||
(*FreeHandler)( p );
|
||||
}
|
||||
}
|
||||
|
||||
bool __TBB_EXPORTED_FUNC is_malloc_used_v3() {
|
||||
if (MallocHandler == &DummyMalloc) {
|
||||
void* void_ptr = (*MallocHandler)(1);
|
||||
(*FreeHandler)(void_ptr);
|
||||
}
|
||||
__TBB_ASSERT( MallocHandler!=&DummyMalloc && FreeHandler!=&DummyFree, NULL );
|
||||
// Cast to void avoids type mismatch errors on some compilers (e.g. __IBMCPP__)
|
||||
__TBB_ASSERT( !(((void*)MallocHandler==(void*)&malloc) ^ ((void*)FreeHandler==(void*)&free)),
|
||||
"Both shim pointers must refer to routines from the same package (either TBB or CRT)" );
|
||||
return (void*)MallocHandler == (void*)&malloc;
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
|
||||
} // namespace tbb
|
||||
@@ -0,0 +1,137 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_cache_aligned_allocator_H
|
||||
#define __TBB_cache_aligned_allocator_H
|
||||
|
||||
#include <new>
|
||||
#include "tbb_stddef.h"
|
||||
#if __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
|
||||
#include <utility> // std::forward
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
|
||||
//! @cond INTERNAL
|
||||
namespace internal {
|
||||
//! Cache/sector line size.
|
||||
/** @ingroup memory_allocation */
|
||||
size_t __TBB_EXPORTED_FUNC NFS_GetLineSize();
|
||||
|
||||
//! Allocate memory on cache/sector line boundary.
|
||||
/** @ingroup memory_allocation */
|
||||
void* __TBB_EXPORTED_FUNC NFS_Allocate( size_t n_element, size_t element_size, void* hint );
|
||||
|
||||
//! Free memory allocated by NFS_Allocate.
|
||||
/** Freeing a NULL pointer is allowed, but has no effect.
|
||||
@ingroup memory_allocation */
|
||||
void __TBB_EXPORTED_FUNC NFS_Free( void* );
|
||||
}
|
||||
//! @endcond
|
||||
|
||||
#if _MSC_VER && !defined(__INTEL_COMPILER)
|
||||
// Workaround for erroneous "unreferenced parameter" warning in method destroy.
|
||||
#pragma warning (push)
|
||||
#pragma warning (disable: 4100)
|
||||
#endif
|
||||
|
||||
//! Meets "allocator" requirements of ISO C++ Standard, Section 20.1.5
|
||||
/** The members are ordered the same way they are in section 20.4.1
|
||||
of the ISO C++ standard.
|
||||
@ingroup memory_allocation */
|
||||
template<typename T>
|
||||
class cache_aligned_allocator {
|
||||
public:
|
||||
typedef typename internal::allocator_type<T>::value_type value_type;
|
||||
typedef value_type* pointer;
|
||||
typedef const value_type* const_pointer;
|
||||
typedef value_type& reference;
|
||||
typedef const value_type& const_reference;
|
||||
typedef size_t size_type;
|
||||
typedef ptrdiff_t difference_type;
|
||||
template<typename U> struct rebind {
|
||||
typedef cache_aligned_allocator<U> other;
|
||||
};
|
||||
|
||||
cache_aligned_allocator() throw() {}
|
||||
cache_aligned_allocator( const cache_aligned_allocator& ) throw() {}
|
||||
template<typename U> cache_aligned_allocator(const cache_aligned_allocator<U>&) throw() {}
|
||||
|
||||
pointer address(reference x) const {return &x;}
|
||||
const_pointer address(const_reference x) const {return &x;}
|
||||
|
||||
//! Allocate space for n objects, starting on a cache/sector line.
|
||||
pointer allocate( size_type n, const void* hint=0 ) {
|
||||
// The "hint" argument is always ignored in NFS_Allocate thus const_cast shouldn't hurt
|
||||
return pointer(internal::NFS_Allocate( n, sizeof(value_type), const_cast<void*>(hint) ));
|
||||
}
|
||||
|
||||
//! Free block of memory that starts on a cache line
|
||||
void deallocate( pointer p, size_type ) {
|
||||
internal::NFS_Free(p);
|
||||
}
|
||||
|
||||
//! Largest value for which method allocate might succeed.
|
||||
size_type max_size() const throw() {
|
||||
return (~size_t(0)-internal::NFS_MaxLineSize)/sizeof(value_type);
|
||||
}
|
||||
|
||||
//! Copy-construct value at location pointed to by p.
|
||||
#if __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
|
||||
template<typename U, typename... Args>
|
||||
void construct(U *p, Args&&... args)
|
||||
{ ::new((void *)p) U(std::forward<Args>(args)...); }
|
||||
#else // __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
void construct( pointer p, value_type&& value ) {::new((void*)(p)) value_type(std::move(value));}
|
||||
#endif
|
||||
void construct( pointer p, const value_type& value ) {::new((void*)(p)) value_type(value);}
|
||||
#endif // __TBB_ALLOCATOR_CONSTRUCT_VARIADIC
|
||||
|
||||
//! Destroy value at location pointed to by p.
|
||||
void destroy( pointer p ) {p->~value_type();}
|
||||
};
|
||||
|
||||
#if _MSC_VER && !defined(__INTEL_COMPILER)
|
||||
#pragma warning (pop)
|
||||
#endif // warning 4100 is back
|
||||
|
||||
//! Analogous to std::allocator<void>, as defined in ISO C++ Standard, Section 20.4.1
|
||||
/** @ingroup memory_allocation */
|
||||
template<>
|
||||
class cache_aligned_allocator<void> {
|
||||
public:
|
||||
typedef void* pointer;
|
||||
typedef const void* const_pointer;
|
||||
typedef void value_type;
|
||||
template<typename U> struct rebind {
|
||||
typedef cache_aligned_allocator<U> other;
|
||||
};
|
||||
};
|
||||
|
||||
template<typename T, typename U>
|
||||
inline bool operator==( const cache_aligned_allocator<T>&, const cache_aligned_allocator<U>& ) {return true;}
|
||||
|
||||
template<typename T, typename U>
|
||||
inline bool operator!=( const cache_aligned_allocator<T>&, const cache_aligned_allocator<U>& ) {return false;}
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_cache_aligned_allocator_H */
|
||||
@@ -0,0 +1,115 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
/* The API to enable interoperability between Intel(R) Cilk(TM) Plus and
|
||||
Intel(R) Threading Building Blocks. */
|
||||
|
||||
#ifndef CILK_TBB_INTEROP_H
|
||||
#define CILK_TBB_INTEROP_H
|
||||
|
||||
#ifndef _WIN32
|
||||
#ifdef IN_CILK_RUNTIME
|
||||
#define CILK_EXPORT __attribute__((visibility("protected")))
|
||||
#else
|
||||
#define CILK_EXPORT /* nothing */
|
||||
#endif
|
||||
#else
|
||||
#ifdef IN_CILK_RUNTIME
|
||||
#define CILK_EXPORT __declspec(dllexport)
|
||||
#else
|
||||
#define CILK_EXPORT __declspec(dllimport)
|
||||
#endif // IN_CILK_RUNTIME
|
||||
#endif // _WIN32
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif /* __cplusplus */
|
||||
|
||||
/* A return code. 0 indicates success */
|
||||
typedef int __cilk_tbb_retcode;
|
||||
|
||||
enum __cilk_tbb_stack_op {
|
||||
CILK_TBB_STACK_ORPHAN, // disconnecting stack from a thread
|
||||
CILK_TBB_STACK_ADOPT, // reconnecting orphaned stack to a trhead
|
||||
CILK_TBB_STACK_RELEASE // releasing stack
|
||||
};
|
||||
|
||||
typedef __cilk_tbb_retcode (*__cilk_tbb_pfn_stack_op)(enum __cilk_tbb_stack_op, void* data);
|
||||
|
||||
typedef __cilk_tbb_retcode (*__cilk_tbb_pfn_unwatch_stacks)(void *data);
|
||||
|
||||
/* Each thunk structure has two pointers: "routine" and "data".
|
||||
The caller of the thunk invokes *routine, passing "data" as the void* parameter. */
|
||||
|
||||
/* Thunk invoked by Intel Cilk Plus runtime (cilkrts) when it changes the relationship
|
||||
between a stack and a thread. It does not matter what stack the thunk runs on.
|
||||
The thread (not fiber) on which the thunk runs is important.
|
||||
|
||||
CILK_TBB_STACK_ORPHAN
|
||||
The thunk must be invoked on the thread disconnecting itself from the stack.
|
||||
Must "happen before" the stack is adopted elsewhere.
|
||||
CILK_TBB_STACK_ADOPT
|
||||
The thunk must be invoked on the thread adopting the stack.
|
||||
CILK_TBB_STACK_RELEASE
|
||||
The thunk must be invoked on the thread doing the releasing,
|
||||
Must "happen before" the stack is used elsewhere.
|
||||
|
||||
When a non-empty stack is transfered between threads, the first thread must orphan it
|
||||
and the second thread must adopt it.
|
||||
|
||||
An empty stack can be transfered similarly, or simply released by the first thread.
|
||||
|
||||
Here is a summary of the actions as transitions on a state machine.
|
||||
|
||||
watch ORPHAN
|
||||
-->--> -->--
|
||||
/ \ / \
|
||||
(freed empty stack) (TBB sees stack running on thread) (stack in limbo)
|
||||
| \ / \ / |
|
||||
| --<-- --<-- |
|
||||
^ RELEASE or ADOPT V
|
||||
\ unwatch /
|
||||
\ /
|
||||
--------------------------<---------------------------
|
||||
RELEASE
|
||||
*/
|
||||
struct __cilk_tbb_stack_op_thunk {
|
||||
__cilk_tbb_pfn_stack_op routine;
|
||||
void* data; /* Set by TBB */
|
||||
};
|
||||
|
||||
/* Thunk invoked by TBB when it is no longer interested in watching the stack bound to the current thread. */
|
||||
struct __cilk_tbb_unwatch_thunk {
|
||||
__cilk_tbb_pfn_unwatch_stacks routine;
|
||||
void* data;
|
||||
};
|
||||
|
||||
/* Defined by cilkrts, called by TBB.
|
||||
Requests that cilkrts invoke __cilk_tbb_stack_op_thunk when it orphans a stack.
|
||||
cilkrts sets *u to a thunk that TBB should call when it is no longer interested in watching the stack. */
|
||||
CILK_EXPORT
|
||||
__cilk_tbb_retcode __cilkrts_watch_stack(struct __cilk_tbb_unwatch_thunk* u,
|
||||
struct __cilk_tbb_stack_op_thunk o);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif /* __cplusplus */
|
||||
|
||||
#endif // CILK_TBB_INTEROP_H
|
||||
@@ -0,0 +1,72 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_combinable_H
|
||||
#define __TBB_combinable_H
|
||||
|
||||
#include "enumerable_thread_specific.h"
|
||||
#include "cache_aligned_allocator.h"
|
||||
|
||||
namespace tbb {
|
||||
/** \name combinable
|
||||
**/
|
||||
//@{
|
||||
//! Thread-local storage with optional reduction
|
||||
/** @ingroup containers */
|
||||
template <typename T>
|
||||
class combinable {
|
||||
private:
|
||||
typedef typename tbb::cache_aligned_allocator<T> my_alloc;
|
||||
|
||||
typedef typename tbb::enumerable_thread_specific<T, my_alloc, ets_no_key> my_ets_type;
|
||||
my_ets_type my_ets;
|
||||
|
||||
public:
|
||||
|
||||
combinable() { }
|
||||
|
||||
template <typename finit>
|
||||
combinable( finit _finit) : my_ets(_finit) { }
|
||||
|
||||
//! destructor
|
||||
~combinable() {
|
||||
}
|
||||
|
||||
combinable(const combinable& other) : my_ets(other.my_ets) { }
|
||||
|
||||
combinable & operator=( const combinable & other) { my_ets = other.my_ets; return *this; }
|
||||
|
||||
void clear() { my_ets.clear(); }
|
||||
|
||||
T& local() { return my_ets.local(); }
|
||||
|
||||
T& local(bool & exists) { return my_ets.local(exists); }
|
||||
|
||||
// combine_func_t has signature T(T,T) or T(const T&, const T&)
|
||||
template <typename combine_func_t>
|
||||
T combine(combine_func_t f_combine) { return my_ets.combine(f_combine); }
|
||||
|
||||
// combine_func_t has signature void(T) or void(const T&)
|
||||
template <typename combine_func_t>
|
||||
void combine_each(combine_func_t f_combine) { my_ets.combine_each(f_combine); }
|
||||
|
||||
};
|
||||
} // namespace tbb
|
||||
#endif /* __TBB_combinable_H */
|
||||
@@ -0,0 +1,457 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_condition_variable_H
|
||||
#define __TBB_condition_variable_H
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#include "../machine/windows_api.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace interface5 {
|
||||
namespace internal {
|
||||
struct condition_variable_using_event
|
||||
{
|
||||
//! Event for blocking waiting threads.
|
||||
HANDLE event;
|
||||
//! Protects invariants involving n_waiters, release_count, and epoch.
|
||||
CRITICAL_SECTION mutex;
|
||||
//! Number of threads waiting on this condition variable
|
||||
int n_waiters;
|
||||
//! Number of threads remaining that should no longer wait on this condition variable.
|
||||
int release_count;
|
||||
//! To keep threads from waking up prematurely with earlier signals.
|
||||
unsigned epoch;
|
||||
};
|
||||
}}} // namespace tbb::interface5::internal
|
||||
|
||||
#ifndef CONDITION_VARIABLE_INIT
|
||||
typedef void* CONDITION_VARIABLE;
|
||||
typedef CONDITION_VARIABLE* PCONDITION_VARIABLE;
|
||||
#endif
|
||||
|
||||
#else /* if not _WIN32||_WIN64 */
|
||||
#include <errno.h> // some systems need it for ETIMEDOUT
|
||||
#include <pthread.h>
|
||||
#if __linux__
|
||||
#include <ctime>
|
||||
#else /* generic Unix */
|
||||
#include <sys/time.h>
|
||||
#endif
|
||||
#endif /* _WIN32||_WIN64 */
|
||||
|
||||
#include "../tbb_stddef.h"
|
||||
#include "../mutex.h"
|
||||
#include "../tbb_thread.h"
|
||||
#include "../tbb_exception.h"
|
||||
#include "../tbb_profiling.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace interface5 {
|
||||
|
||||
// C++0x standard working draft 30.4.3
|
||||
// Lock tag types
|
||||
struct defer_lock_t { }; //! do not acquire ownership of the mutex
|
||||
struct try_to_lock_t { }; //! try to acquire ownership of the mutex without blocking
|
||||
struct adopt_lock_t { }; //! assume the calling thread has already
|
||||
const defer_lock_t defer_lock = {};
|
||||
const try_to_lock_t try_to_lock = {};
|
||||
const adopt_lock_t adopt_lock = {};
|
||||
|
||||
// C++0x standard working draft 30.4.3.1
|
||||
//! lock_guard
|
||||
template<typename M>
|
||||
class lock_guard : tbb::internal::no_copy {
|
||||
public:
|
||||
//! mutex type
|
||||
typedef M mutex_type;
|
||||
|
||||
//! Constructor
|
||||
/** precondition: If mutex_type is not a recursive mutex, the calling thread
|
||||
does not own the mutex m. */
|
||||
explicit lock_guard(mutex_type& m) : pm(m) {m.lock();}
|
||||
|
||||
//! Adopt_lock constructor
|
||||
/** precondition: the calling thread owns the mutex m. */
|
||||
lock_guard(mutex_type& m, adopt_lock_t) : pm(m) {}
|
||||
|
||||
//! Destructor
|
||||
~lock_guard() { pm.unlock(); }
|
||||
private:
|
||||
mutex_type& pm;
|
||||
};
|
||||
|
||||
// C++0x standard working draft 30.4.3.2
|
||||
//! unique_lock
|
||||
template<typename M>
|
||||
class unique_lock : tbb::internal::no_copy {
|
||||
friend class condition_variable;
|
||||
public:
|
||||
typedef M mutex_type;
|
||||
|
||||
// 30.4.3.2.1 construct/copy/destroy
|
||||
// NB: Without constructors that take an r-value reference to a unique_lock, the following constructor is of little use.
|
||||
//! Constructor
|
||||
/** postcondition: pm==0 && owns==false */
|
||||
unique_lock() : pm(NULL), owns(false) {}
|
||||
|
||||
//! Constructor
|
||||
/** precondition: if mutex_type is not a recursive mutex, the calling thread
|
||||
does not own the mutex m. If the precondition is not met, a deadlock occurs.
|
||||
postcondition: pm==&m and owns==true */
|
||||
explicit unique_lock(mutex_type& m) : pm(&m) {m.lock(); owns=true;}
|
||||
|
||||
//! Defer_lock constructor
|
||||
/** postcondition: pm==&m and owns==false */
|
||||
unique_lock(mutex_type& m, defer_lock_t) : pm(&m), owns(false) {}
|
||||
|
||||
//! Try_to_lock constructor
|
||||
/** precondition: if mutex_type is not a recursive mutex, the calling thread
|
||||
does not own the mutex m. If the precondition is not met, a deadlock occurs.
|
||||
postcondition: pm==&m and owns==res where res is the value returned by
|
||||
the call to m.try_lock(). */
|
||||
unique_lock(mutex_type& m, try_to_lock_t) : pm(&m) {owns = m.try_lock();}
|
||||
|
||||
//! Adopt_lock constructor
|
||||
/** precondition: the calling thread owns the mutex. If it does not, mutex->unlock() would fail.
|
||||
postcondition: pm==&m and owns==true */
|
||||
unique_lock(mutex_type& m, adopt_lock_t) : pm(&m), owns(true) {}
|
||||
|
||||
//! Timed unique_lock acquisition.
|
||||
/** To avoid requiring support for namespace chrono, this method deviates from the working draft in that
|
||||
it uses tbb::tick_count::interval_t to specify the time duration. */
|
||||
unique_lock(mutex_type& m, const tick_count::interval_t &i) : pm(&m) {owns = try_lock_for( i );}
|
||||
|
||||
//! Destructor
|
||||
~unique_lock() { if( owns ) pm->unlock(); }
|
||||
|
||||
// 30.4.3.2.2 locking
|
||||
//! Lock the mutex and own it.
|
||||
void lock() {
|
||||
if( pm ) {
|
||||
if( !owns ) {
|
||||
pm->lock();
|
||||
owns = true;
|
||||
} else
|
||||
throw_exception_v4( tbb::internal::eid_possible_deadlock );
|
||||
} else
|
||||
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
|
||||
__TBB_ASSERT( owns, NULL );
|
||||
}
|
||||
|
||||
//! Try to lock the mutex.
|
||||
/** If successful, note that this lock owns it. Otherwise, set it false. */
|
||||
bool try_lock() {
|
||||
if( pm ) {
|
||||
if( !owns )
|
||||
owns = pm->try_lock();
|
||||
else
|
||||
throw_exception_v4( tbb::internal::eid_possible_deadlock );
|
||||
} else
|
||||
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
|
||||
return owns;
|
||||
}
|
||||
|
||||
//! Try to lock the mutex.
|
||||
bool try_lock_for( const tick_count::interval_t &i );
|
||||
|
||||
//! Unlock the mutex
|
||||
/** And note that this lock no longer owns it. */
|
||||
void unlock() {
|
||||
if( owns ) {
|
||||
pm->unlock();
|
||||
owns = false;
|
||||
} else
|
||||
throw_exception_v4( tbb::internal::eid_operation_not_permitted );
|
||||
__TBB_ASSERT( !owns, NULL );
|
||||
}
|
||||
|
||||
// 30.4.3.2.3 modifiers
|
||||
//! Swap the two unique locks
|
||||
void swap(unique_lock& u) {
|
||||
mutex_type* t_pm = u.pm; u.pm = pm; pm = t_pm;
|
||||
bool t_owns = u.owns; u.owns = owns; owns = t_owns;
|
||||
}
|
||||
|
||||
//! Release control over the mutex.
|
||||
mutex_type* release() {
|
||||
mutex_type* o_pm = pm;
|
||||
pm = NULL;
|
||||
owns = false;
|
||||
return o_pm;
|
||||
}
|
||||
|
||||
// 30.4.3.2.4 observers
|
||||
//! Does this lock own the mutex?
|
||||
bool owns_lock() const { return owns; }
|
||||
|
||||
// TODO: Un-comment 'explicit' when the last non-C++0x compiler support is dropped
|
||||
//! Does this lock own the mutex?
|
||||
/*explicit*/ operator bool() const { return owns; }
|
||||
|
||||
//! Return the mutex that this lock currently has.
|
||||
mutex_type* mutex() const { return pm; }
|
||||
|
||||
private:
|
||||
mutex_type* pm;
|
||||
bool owns;
|
||||
};
|
||||
|
||||
template<typename M>
|
||||
bool unique_lock<M>::try_lock_for( const tick_count::interval_t &i)
|
||||
{
|
||||
const int unique_lock_tick = 100; /* microseconds; 0.1 milliseconds */
|
||||
// the smallest wait-time is 0.1 milliseconds.
|
||||
bool res = pm->try_lock();
|
||||
int duration_in_micro;
|
||||
if( !res && (duration_in_micro=int(i.seconds()*1e6))>unique_lock_tick ) {
|
||||
tick_count::interval_t i_100( double(unique_lock_tick)/1e6 /* seconds */); // 100 microseconds = 0.1*10E-3
|
||||
do {
|
||||
this_tbb_thread::sleep(i_100); // sleep for 100 micro seconds
|
||||
duration_in_micro -= unique_lock_tick;
|
||||
res = pm->try_lock();
|
||||
} while( !res && duration_in_micro>unique_lock_tick );
|
||||
}
|
||||
return (owns=res);
|
||||
}
|
||||
|
||||
//! Swap the two unique locks that have the mutexes of same type
|
||||
template<typename M>
|
||||
void swap(unique_lock<M>& x, unique_lock<M>& y) { x.swap( y ); }
|
||||
|
||||
namespace internal {
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
union condvar_impl_t {
|
||||
condition_variable_using_event cv_event;
|
||||
CONDITION_VARIABLE cv_native;
|
||||
};
|
||||
void __TBB_EXPORTED_FUNC internal_initialize_condition_variable( condvar_impl_t& cv );
|
||||
void __TBB_EXPORTED_FUNC internal_destroy_condition_variable( condvar_impl_t& cv );
|
||||
void __TBB_EXPORTED_FUNC internal_condition_variable_notify_one( condvar_impl_t& cv );
|
||||
void __TBB_EXPORTED_FUNC internal_condition_variable_notify_all( condvar_impl_t& cv );
|
||||
bool __TBB_EXPORTED_FUNC internal_condition_variable_wait( condvar_impl_t& cv, mutex* mtx, const tick_count::interval_t* i = NULL );
|
||||
|
||||
#else /* if !(_WIN32||_WIN64), i.e., POSIX threads */
|
||||
typedef pthread_cond_t condvar_impl_t;
|
||||
#endif
|
||||
|
||||
} // namespace internal
|
||||
|
||||
//! cv_status
|
||||
/** C++0x standard working draft 30.5 */
|
||||
enum cv_status { no_timeout, timeout };
|
||||
|
||||
//! condition variable
|
||||
/** C++0x standard working draft 30.5.1
|
||||
@ingroup synchronization */
|
||||
class condition_variable : tbb::internal::no_copy {
|
||||
public:
|
||||
//! Constructor
|
||||
condition_variable() {
|
||||
#if _WIN32||_WIN64
|
||||
internal_initialize_condition_variable( my_cv );
|
||||
#else
|
||||
pthread_cond_init( &my_cv, NULL );
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Destructor
|
||||
~condition_variable() {
|
||||
//precondition: There shall be no thread blocked on *this.
|
||||
#if _WIN32||_WIN64
|
||||
internal_destroy_condition_variable( my_cv );
|
||||
#else
|
||||
pthread_cond_destroy( &my_cv );
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Notify one thread and wake it up
|
||||
void notify_one() {
|
||||
#if _WIN32||_WIN64
|
||||
internal_condition_variable_notify_one( my_cv );
|
||||
#else
|
||||
pthread_cond_signal( &my_cv );
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Notify all threads
|
||||
void notify_all() {
|
||||
#if _WIN32||_WIN64
|
||||
internal_condition_variable_notify_all( my_cv );
|
||||
#else
|
||||
pthread_cond_broadcast( &my_cv );
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Release the mutex associated with the lock and wait on this condition variable
|
||||
void wait(unique_lock<mutex>& lock);
|
||||
|
||||
//! Wait on this condition variable while pred is false
|
||||
template <class Predicate>
|
||||
void wait(unique_lock<mutex>& lock, Predicate pred) {
|
||||
while( !pred() )
|
||||
wait( lock );
|
||||
}
|
||||
|
||||
//! Timed version of wait()
|
||||
cv_status wait_for(unique_lock<mutex>& lock, const tick_count::interval_t &i );
|
||||
|
||||
//! Timed version of the predicated wait
|
||||
/** The loop terminates when pred() returns true or when the time duration specified by rel_time (i) has elapsed. */
|
||||
template<typename Predicate>
|
||||
bool wait_for(unique_lock<mutex>& lock, const tick_count::interval_t &i, Predicate pred)
|
||||
{
|
||||
while( !pred() ) {
|
||||
cv_status st = wait_for( lock, i );
|
||||
if( st==timeout )
|
||||
return pred();
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// C++0x standard working draft. 30.2.3
|
||||
typedef internal::condvar_impl_t* native_handle_type;
|
||||
|
||||
native_handle_type native_handle() { return (native_handle_type) &my_cv; }
|
||||
|
||||
private:
|
||||
internal::condvar_impl_t my_cv;
|
||||
};
|
||||
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
inline void condition_variable::wait( unique_lock<mutex>& lock )
|
||||
{
|
||||
__TBB_ASSERT( lock.owns, NULL );
|
||||
lock.owns = false;
|
||||
if( !internal_condition_variable_wait( my_cv, lock.mutex() ) ) {
|
||||
int ec = GetLastError();
|
||||
// on Windows 7, SleepConditionVariableCS() may return ERROR_TIMEOUT while the doc says it returns WAIT_TIMEOUT
|
||||
__TBB_ASSERT_EX( ec!=WAIT_TIMEOUT&&ec!=ERROR_TIMEOUT, NULL );
|
||||
lock.owns = true;
|
||||
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
|
||||
}
|
||||
lock.owns = true;
|
||||
}
|
||||
|
||||
inline cv_status condition_variable::wait_for( unique_lock<mutex>& lock, const tick_count::interval_t& i )
|
||||
{
|
||||
cv_status rc = no_timeout;
|
||||
__TBB_ASSERT( lock.owns, NULL );
|
||||
lock.owns = false;
|
||||
// condvar_wait could be SleepConditionVariableCS (or SleepConditionVariableSRW) or our own pre-vista cond_var_wait()
|
||||
if( !internal_condition_variable_wait( my_cv, lock.mutex(), &i ) ) {
|
||||
int ec = GetLastError();
|
||||
if( ec==WAIT_TIMEOUT || ec==ERROR_TIMEOUT )
|
||||
rc = timeout;
|
||||
else {
|
||||
lock.owns = true;
|
||||
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
|
||||
}
|
||||
}
|
||||
lock.owns = true;
|
||||
return rc;
|
||||
}
|
||||
|
||||
#else /* !(_WIN32||_WIN64) */
|
||||
inline void condition_variable::wait( unique_lock<mutex>& lock )
|
||||
{
|
||||
__TBB_ASSERT( lock.owns, NULL );
|
||||
lock.owns = false;
|
||||
if( pthread_cond_wait( &my_cv, lock.mutex()->native_handle() ) ) {
|
||||
lock.owns = true;
|
||||
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
|
||||
}
|
||||
// upon successful return, the mutex has been locked and is owned by the calling thread.
|
||||
lock.owns = true;
|
||||
}
|
||||
|
||||
inline cv_status condition_variable::wait_for( unique_lock<mutex>& lock, const tick_count::interval_t& i )
|
||||
{
|
||||
#if __linux__
|
||||
struct timespec req;
|
||||
double sec = i.seconds();
|
||||
clock_gettime( CLOCK_REALTIME, &req );
|
||||
req.tv_sec += static_cast<long>(sec);
|
||||
req.tv_nsec += static_cast<long>( (sec - static_cast<long>(sec))*1e9 );
|
||||
#else /* generic Unix */
|
||||
struct timeval tv;
|
||||
struct timespec req;
|
||||
double sec = i.seconds();
|
||||
int status = gettimeofday(&tv, NULL);
|
||||
__TBB_ASSERT_EX( status==0, "gettimeofday failed" );
|
||||
req.tv_sec = tv.tv_sec + static_cast<long>(sec);
|
||||
req.tv_nsec = tv.tv_usec*1000 + static_cast<long>( (sec - static_cast<long>(sec))*1e9 );
|
||||
#endif /*(choice of OS) */
|
||||
if( req.tv_nsec>=1e9 ) {
|
||||
req.tv_sec += 1;
|
||||
req.tv_nsec -= static_cast<long int>(1e9);
|
||||
}
|
||||
__TBB_ASSERT( 0<=req.tv_nsec && req.tv_nsec<1e9, NULL );
|
||||
|
||||
int ec;
|
||||
cv_status rc = no_timeout;
|
||||
__TBB_ASSERT( lock.owns, NULL );
|
||||
lock.owns = false;
|
||||
if( ( ec=pthread_cond_timedwait( &my_cv, lock.mutex()->native_handle(), &req ) ) ) {
|
||||
if( ec==ETIMEDOUT )
|
||||
rc = timeout;
|
||||
else {
|
||||
__TBB_ASSERT( lock.try_lock()==false, NULL );
|
||||
lock.owns = true;
|
||||
throw_exception_v4( tbb::internal::eid_condvar_wait_failed );
|
||||
}
|
||||
}
|
||||
lock.owns = true;
|
||||
return rc;
|
||||
}
|
||||
#endif /* !(_WIN32||_WIN64) */
|
||||
|
||||
} // namespace interface5
|
||||
|
||||
__TBB_DEFINE_PROFILING_SET_NAME(interface5::condition_variable)
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#if TBB_IMPLEMENT_CPP0X
|
||||
|
||||
namespace std {
|
||||
|
||||
using tbb::interface5::defer_lock_t;
|
||||
using tbb::interface5::try_to_lock_t;
|
||||
using tbb::interface5::adopt_lock_t;
|
||||
using tbb::interface5::defer_lock;
|
||||
using tbb::interface5::try_to_lock;
|
||||
using tbb::interface5::adopt_lock;
|
||||
using tbb::interface5::lock_guard;
|
||||
using tbb::interface5::unique_lock;
|
||||
using tbb::interface5::swap; /* this is for void std::swap(unique_lock<M>&,unique_lock<M>&) */
|
||||
using tbb::interface5::condition_variable;
|
||||
using tbb::interface5::cv_status;
|
||||
using tbb::interface5::timeout;
|
||||
using tbb::interface5::no_timeout;
|
||||
|
||||
} // namespace std
|
||||
|
||||
#endif /* TBB_IMPLEMENT_CPP0X */
|
||||
|
||||
#endif /* __TBB_condition_variable_H */
|
||||
@@ -0,0 +1,62 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_compat_ppl_H
|
||||
#define __TBB_compat_ppl_H
|
||||
|
||||
#include "../task_group.h"
|
||||
#include "../parallel_invoke.h"
|
||||
#include "../parallel_for_each.h"
|
||||
#include "../parallel_for.h"
|
||||
#include "../tbb_exception.h"
|
||||
#include "../critical_section.h"
|
||||
#include "../reader_writer_lock.h"
|
||||
#include "../combinable.h"
|
||||
|
||||
namespace Concurrency {
|
||||
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
using tbb::task_handle;
|
||||
using tbb::task_group_status;
|
||||
using tbb::task_group;
|
||||
using tbb::structured_task_group;
|
||||
using tbb::invalid_multiple_scheduling;
|
||||
using tbb::missing_wait;
|
||||
using tbb::make_task;
|
||||
|
||||
using tbb::not_complete;
|
||||
using tbb::complete;
|
||||
using tbb::canceled;
|
||||
|
||||
using tbb::is_current_task_group_canceling;
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
using tbb::parallel_invoke;
|
||||
using tbb::strict_ppl::parallel_for;
|
||||
using tbb::parallel_for_each;
|
||||
using tbb::critical_section;
|
||||
using tbb::reader_writer_lock;
|
||||
using tbb::combinable;
|
||||
|
||||
using tbb::improper_lock;
|
||||
|
||||
} // namespace Concurrency
|
||||
|
||||
#endif /* __TBB_compat_ppl_H */
|
||||
@@ -0,0 +1,46 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_thread_H
|
||||
#define __TBB_thread_H
|
||||
|
||||
#include "../tbb_thread.h"
|
||||
|
||||
#if TBB_IMPLEMENT_CPP0X
|
||||
|
||||
namespace std {
|
||||
|
||||
typedef tbb::tbb_thread thread;
|
||||
|
||||
namespace this_thread {
|
||||
using tbb::this_tbb_thread::get_id;
|
||||
using tbb::this_tbb_thread::yield;
|
||||
|
||||
inline void sleep_for(const tbb::tick_count::interval_t& rel_time) {
|
||||
tbb::internal::thread_sleep_v3( rel_time );
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
#endif /* TBB_IMPLEMENT_CPP0X */
|
||||
|
||||
#endif /* __TBB_thread_H */
|
||||
488
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/compat/tuple
vendored
Normal file
488
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/compat/tuple
vendored
Normal file
@@ -0,0 +1,488 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_tuple_H
|
||||
#define __TBB_tuple_H
|
||||
|
||||
#include <utility>
|
||||
#include "../tbb_stddef.h"
|
||||
|
||||
// build preprocessor variables for varying number of arguments
|
||||
// Need the leading comma so the empty __TBB_T_PACK will not cause a syntax error.
|
||||
#if __TBB_VARIADIC_MAX <= 5
|
||||
#define __TBB_T_PACK
|
||||
#define __TBB_U_PACK
|
||||
#define __TBB_TYPENAME_T_PACK
|
||||
#define __TBB_TYPENAME_U_PACK
|
||||
#define __TBB_NULL_TYPE_PACK
|
||||
#define __TBB_REF_T_PARAM_PACK
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK
|
||||
#define __TBB_T_PARAM_LIST_PACK
|
||||
#define __TBB_CONST_NULL_REF_PACK
|
||||
//
|
||||
#elif __TBB_VARIADIC_MAX == 6
|
||||
#define __TBB_T_PACK ,__T5
|
||||
#define __TBB_U_PACK ,__U5
|
||||
#define __TBB_TYPENAME_T_PACK , typename __T5
|
||||
#define __TBB_TYPENAME_U_PACK , typename __U5
|
||||
#define __TBB_NULL_TYPE_PACK , null_type
|
||||
#define __TBB_REF_T_PARAM_PACK ,__T5& t5
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK ,const __T5& t5
|
||||
#define __TBB_T_PARAM_LIST_PACK ,t5
|
||||
#define __TBB_CONST_NULL_REF_PACK , const null_type&
|
||||
//
|
||||
#elif __TBB_VARIADIC_MAX == 7
|
||||
#define __TBB_T_PACK ,__T5, __T6
|
||||
#define __TBB_U_PACK ,__U5, __U6
|
||||
#define __TBB_TYPENAME_T_PACK , typename __T5 , typename __T6
|
||||
#define __TBB_TYPENAME_U_PACK , typename __U5 , typename __U6
|
||||
#define __TBB_NULL_TYPE_PACK , null_type, null_type
|
||||
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK ,const __T5& t5, const __T6& t6
|
||||
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6
|
||||
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&
|
||||
//
|
||||
#elif __TBB_VARIADIC_MAX == 8
|
||||
#define __TBB_T_PACK ,__T5, __T6, __T7
|
||||
#define __TBB_U_PACK ,__U5, __U6, __U7
|
||||
#define __TBB_TYPENAME_T_PACK , typename __T5 , typename __T6, typename __T7
|
||||
#define __TBB_TYPENAME_U_PACK , typename __U5 , typename __U6, typename __U7
|
||||
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type
|
||||
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7
|
||||
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7
|
||||
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&
|
||||
//
|
||||
#elif __TBB_VARIADIC_MAX == 9
|
||||
#define __TBB_T_PACK ,__T5, __T6, __T7, __T8
|
||||
#define __TBB_U_PACK ,__U5, __U6, __U7, __U8
|
||||
#define __TBB_TYPENAME_T_PACK , typename __T5, typename __T6, typename __T7, typename __T8
|
||||
#define __TBB_TYPENAME_U_PACK , typename __U5, typename __U6, typename __U7, typename __U8
|
||||
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type, null_type
|
||||
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7, __T8& t8
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7, const __T8& t8
|
||||
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7 ,t8
|
||||
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&, const null_type&
|
||||
//
|
||||
#elif __TBB_VARIADIC_MAX >= 10
|
||||
#define __TBB_T_PACK ,__T5, __T6, __T7, __T8, __T9
|
||||
#define __TBB_U_PACK ,__U5, __U6, __U7, __U8, __U9
|
||||
#define __TBB_TYPENAME_T_PACK , typename __T5, typename __T6, typename __T7, typename __T8, typename __T9
|
||||
#define __TBB_TYPENAME_U_PACK , typename __U5, typename __U6, typename __U7, typename __U8, typename __U9
|
||||
#define __TBB_NULL_TYPE_PACK , null_type, null_type, null_type, null_type, null_type
|
||||
#define __TBB_REF_T_PARAM_PACK ,__T5& t5, __T6& t6, __T7& t7, __T8& t8, __T9& t9
|
||||
#define __TBB_CONST_REF_T_PARAM_PACK , const __T5& t5, const __T6& t6, const __T7& t7, const __T8& t8, const __T9& t9
|
||||
#define __TBB_T_PARAM_LIST_PACK ,t5 ,t6 ,t7 ,t8 ,t9
|
||||
#define __TBB_CONST_NULL_REF_PACK , const null_type&, const null_type&, const null_type&, const null_type&, const null_type&
|
||||
#endif
|
||||
|
||||
|
||||
|
||||
namespace tbb {
|
||||
namespace interface5 {
|
||||
|
||||
namespace internal {
|
||||
struct null_type { };
|
||||
}
|
||||
using internal::null_type;
|
||||
|
||||
// tuple forward declaration
|
||||
template <typename __T0=null_type, typename __T1=null_type, typename __T2=null_type,
|
||||
typename __T3=null_type, typename __T4=null_type
|
||||
#if __TBB_VARIADIC_MAX >= 6
|
||||
, typename __T5=null_type
|
||||
#if __TBB_VARIADIC_MAX >= 7
|
||||
, typename __T6=null_type
|
||||
#if __TBB_VARIADIC_MAX >= 8
|
||||
, typename __T7=null_type
|
||||
#if __TBB_VARIADIC_MAX >= 9
|
||||
, typename __T8=null_type
|
||||
#if __TBB_VARIADIC_MAX >= 10
|
||||
, typename __T9=null_type
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
>
|
||||
class tuple;
|
||||
|
||||
namespace internal {
|
||||
|
||||
// const null_type temp
|
||||
inline const null_type cnull() { return null_type(); }
|
||||
|
||||
// cons forward declaration
|
||||
template <typename __HT, typename __TT> struct cons;
|
||||
|
||||
// type of a component of the cons
|
||||
template<int __N, typename __T>
|
||||
struct component {
|
||||
typedef typename __T::tail_type next;
|
||||
typedef typename component<__N-1,next>::type type;
|
||||
};
|
||||
|
||||
template<typename __T>
|
||||
struct component<0,__T> {
|
||||
typedef typename __T::head_type type;
|
||||
};
|
||||
|
||||
template<>
|
||||
struct component<0,null_type> {
|
||||
typedef null_type type;
|
||||
};
|
||||
|
||||
// const version of component
|
||||
|
||||
template<int __N, typename __T>
|
||||
struct component<__N, const __T>
|
||||
{
|
||||
typedef typename __T::tail_type next;
|
||||
typedef const typename component<__N-1,next>::type type;
|
||||
};
|
||||
|
||||
template<typename __T>
|
||||
struct component<0, const __T>
|
||||
{
|
||||
typedef const typename __T::head_type type;
|
||||
};
|
||||
|
||||
|
||||
// helper class for getting components of cons
|
||||
template< int __N>
|
||||
struct get_helper {
|
||||
template<typename __HT, typename __TT>
|
||||
inline static typename component<__N, cons<__HT,__TT> >::type& get(cons<__HT,__TT>& ti) {
|
||||
return get_helper<__N-1>::get(ti.tail);
|
||||
}
|
||||
template<typename __HT, typename __TT>
|
||||
inline static typename component<__N, cons<__HT,__TT> >::type const& get(const cons<__HT,__TT>& ti) {
|
||||
return get_helper<__N-1>::get(ti.tail);
|
||||
}
|
||||
};
|
||||
|
||||
template<>
|
||||
struct get_helper<0> {
|
||||
template<typename __HT, typename __TT>
|
||||
inline static typename component<0, cons<__HT,__TT> >::type& get(cons<__HT,__TT>& ti) {
|
||||
return ti.head;
|
||||
}
|
||||
template<typename __HT, typename __TT>
|
||||
inline static typename component<0, cons<__HT,__TT> >::type const& get(const cons<__HT,__TT>& ti) {
|
||||
return ti.head;
|
||||
}
|
||||
};
|
||||
|
||||
// traits adaptor
|
||||
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK>
|
||||
struct tuple_traits {
|
||||
typedef cons <__T0, typename tuple_traits<__T1, __T2, __T3, __T4 __TBB_T_PACK , null_type>::U > U;
|
||||
};
|
||||
|
||||
template <typename __T0>
|
||||
struct tuple_traits<__T0, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > {
|
||||
typedef cons<__T0, null_type> U;
|
||||
};
|
||||
|
||||
template<>
|
||||
struct tuple_traits<null_type, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > {
|
||||
typedef null_type U;
|
||||
};
|
||||
|
||||
|
||||
// core cons defs
|
||||
template <typename __HT, typename __TT>
|
||||
struct cons{
|
||||
|
||||
typedef __HT head_type;
|
||||
typedef __TT tail_type;
|
||||
|
||||
head_type head;
|
||||
tail_type tail;
|
||||
|
||||
static const int length = 1 + tail_type::length;
|
||||
|
||||
// default constructors
|
||||
explicit cons() : head(), tail() { }
|
||||
|
||||
// non-default constructors
|
||||
cons(head_type& h, const tail_type& t) : head(h), tail(t) { }
|
||||
|
||||
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
|
||||
cons(const __T0& t0, const __T1& t1, const __T2& t2, const __T3& t3, const __T4& t4 __TBB_CONST_REF_T_PARAM_PACK) :
|
||||
head(t0), tail(t1, t2, t3, t4 __TBB_T_PARAM_LIST_PACK, cnull()) { }
|
||||
|
||||
template <typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
|
||||
cons(__T0& t0, __T1& t1, __T2& t2, __T3& t3, __T4& t4 __TBB_REF_T_PARAM_PACK) :
|
||||
head(t0), tail(t1, t2, t3, t4 __TBB_T_PARAM_LIST_PACK , cnull()) { }
|
||||
|
||||
template <typename __HT1, typename __TT1>
|
||||
cons(const cons<__HT1,__TT1>& other) : head(other.head), tail(other.tail) { }
|
||||
|
||||
cons& operator=(const cons& other) { head = other.head; tail = other.tail; return *this; }
|
||||
|
||||
friend bool operator==(const cons& me, const cons& other) {
|
||||
return me.head == other.head && me.tail == other.tail;
|
||||
}
|
||||
friend bool operator<(const cons& me, const cons& other) {
|
||||
return me.head < other.head || (!(other.head < me.head) && me.tail < other.tail);
|
||||
}
|
||||
friend bool operator>(const cons& me, const cons& other) { return other<me; }
|
||||
friend bool operator!=(const cons& me, const cons& other) { return !(me==other); }
|
||||
friend bool operator>=(const cons& me, const cons& other) { return !(me<other); }
|
||||
friend bool operator<=(const cons& me, const cons& other) { return !(me>other); }
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator==(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) {
|
||||
return me.head == other.head && me.tail == other.tail;
|
||||
}
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator<(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) {
|
||||
return me.head < other.head || (!(other.head < me.head) && me.tail < other.tail);
|
||||
}
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator>(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return other<me; }
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator!=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me==other); }
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator>=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me<other); }
|
||||
|
||||
template<typename __HT1, typename __TT1>
|
||||
friend bool operator<=(const cons<__HT,__TT>& me, const cons<__HT1,__TT1>& other) { return !(me>other); }
|
||||
|
||||
|
||||
}; // cons
|
||||
|
||||
|
||||
template <typename __HT>
|
||||
struct cons<__HT,null_type> {
|
||||
|
||||
typedef __HT head_type;
|
||||
typedef null_type tail_type;
|
||||
|
||||
head_type head;
|
||||
|
||||
static const int length = 1;
|
||||
|
||||
// default constructor
|
||||
cons() : head() { /*std::cout << "default constructor 1\n";*/ }
|
||||
|
||||
cons(const null_type&, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head() { /*std::cout << "default constructor 2\n";*/ }
|
||||
|
||||
// non-default constructor
|
||||
template<typename __T1>
|
||||
cons(__T1& t1, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(t1) { /*std::cout << "non-default a1, t1== " << t1 << "\n";*/}
|
||||
|
||||
cons(head_type& h, const null_type& = null_type() ) : head(h) { }
|
||||
cons(const head_type& t0, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(t0) { }
|
||||
|
||||
// converting constructor
|
||||
template<typename __HT1>
|
||||
cons(__HT1 h1, const null_type&, const null_type&, const null_type&, const null_type& __TBB_CONST_NULL_REF_PACK) : head(h1) { }
|
||||
|
||||
// copy constructor
|
||||
template<typename __HT1>
|
||||
cons( const cons<__HT1, null_type>& other) : head(other.head) { }
|
||||
|
||||
// assignment operator
|
||||
cons& operator=(const cons& other) { head = other.head; return *this; }
|
||||
|
||||
friend bool operator==(const cons& me, const cons& other) { return me.head == other.head; }
|
||||
friend bool operator<(const cons& me, const cons& other) { return me.head < other.head; }
|
||||
friend bool operator>(const cons& me, const cons& other) { return other<me; }
|
||||
friend bool operator!=(const cons& me, const cons& other) {return !(me==other); }
|
||||
friend bool operator<=(const cons& me, const cons& other) {return !(me>other); }
|
||||
friend bool operator>=(const cons& me, const cons& other) {return !(me<other); }
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator==(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) {
|
||||
return me.head == other.head;
|
||||
}
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator<(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) {
|
||||
return me.head < other.head;
|
||||
}
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator>(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return other<me; }
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator!=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me==other); }
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator<=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me>other); }
|
||||
|
||||
template<typename __HT1>
|
||||
friend bool operator>=(const cons<__HT,null_type>& me, const cons<__HT1,null_type>& other) { return !(me<other); }
|
||||
|
||||
}; // cons
|
||||
|
||||
template <>
|
||||
struct cons<null_type,null_type> { typedef null_type tail_type; static const int length = 0; };
|
||||
|
||||
// wrapper for default constructor
|
||||
template<typename __T>
|
||||
inline const __T wrap_dcons(__T*) { return __T(); }
|
||||
|
||||
} // namespace internal
|
||||
|
||||
// tuple definition
|
||||
template<typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
|
||||
class tuple : public internal::tuple_traits<__T0, __T1, __T2, __T3, __T4 __TBB_T_PACK >::U {
|
||||
// friends
|
||||
template <typename __T> friend class tuple_size;
|
||||
template<int __N, typename __T> friend struct tuple_element;
|
||||
|
||||
// stl components
|
||||
typedef tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > value_type;
|
||||
typedef value_type *pointer;
|
||||
typedef const value_type *const_pointer;
|
||||
typedef value_type &reference;
|
||||
typedef const value_type &const_reference;
|
||||
typedef size_t size_type;
|
||||
|
||||
typedef typename internal::tuple_traits<__T0,__T1,__T2,__T3, __T4 __TBB_T_PACK >::U my_cons;
|
||||
|
||||
public:
|
||||
tuple(const __T0& t0=internal::wrap_dcons((__T0*)NULL)
|
||||
,const __T1& t1=internal::wrap_dcons((__T1*)NULL)
|
||||
,const __T2& t2=internal::wrap_dcons((__T2*)NULL)
|
||||
,const __T3& t3=internal::wrap_dcons((__T3*)NULL)
|
||||
,const __T4& t4=internal::wrap_dcons((__T4*)NULL)
|
||||
#if __TBB_VARIADIC_MAX >= 6
|
||||
,const __T5& t5=internal::wrap_dcons((__T5*)NULL)
|
||||
#if __TBB_VARIADIC_MAX >= 7
|
||||
,const __T6& t6=internal::wrap_dcons((__T6*)NULL)
|
||||
#if __TBB_VARIADIC_MAX >= 8
|
||||
,const __T7& t7=internal::wrap_dcons((__T7*)NULL)
|
||||
#if __TBB_VARIADIC_MAX >= 9
|
||||
,const __T8& t8=internal::wrap_dcons((__T8*)NULL)
|
||||
#if __TBB_VARIADIC_MAX >= 10
|
||||
,const __T9& t9=internal::wrap_dcons((__T9*)NULL)
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
) :
|
||||
my_cons(t0,t1,t2,t3,t4 __TBB_T_PARAM_LIST_PACK) { }
|
||||
|
||||
template<int __N>
|
||||
struct internal_tuple_element {
|
||||
typedef typename internal::component<__N,my_cons>::type type;
|
||||
};
|
||||
|
||||
template<int __N>
|
||||
typename internal_tuple_element<__N>::type& get() { return internal::get_helper<__N>::get(*this); }
|
||||
|
||||
template<int __N>
|
||||
typename internal_tuple_element<__N>::type const& get() const { return internal::get_helper<__N>::get(*this); }
|
||||
|
||||
template<typename __U1, typename __U2>
|
||||
tuple& operator=(const internal::cons<__U1,__U2>& other) {
|
||||
my_cons::operator=(other);
|
||||
return *this;
|
||||
}
|
||||
|
||||
template<typename __U1, typename __U2>
|
||||
tuple& operator=(const std::pair<__U1,__U2>& other) {
|
||||
// __TBB_ASSERT(tuple_size<value_type>::value == 2, "Invalid size for pair to tuple assignment");
|
||||
this->head = other.first;
|
||||
this->tail.head = other.second;
|
||||
return *this;
|
||||
}
|
||||
|
||||
friend bool operator==(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)==(other);}
|
||||
friend bool operator<(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)<(other);}
|
||||
friend bool operator>(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)>(other);}
|
||||
friend bool operator!=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)!=(other);}
|
||||
friend bool operator>=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)>=(other);}
|
||||
friend bool operator<=(const tuple& me, const tuple& other) {return static_cast<const my_cons &>(me)<=(other);}
|
||||
|
||||
}; // tuple
|
||||
|
||||
// empty tuple
|
||||
template<>
|
||||
class tuple<null_type, null_type, null_type, null_type, null_type __TBB_NULL_TYPE_PACK > : public null_type {
|
||||
};
|
||||
|
||||
// helper classes
|
||||
|
||||
template < typename __T>
|
||||
class tuple_size {
|
||||
public:
|
||||
static const size_t value = 1 + tuple_size<typename __T::tail_type>::value;
|
||||
};
|
||||
|
||||
template <>
|
||||
class tuple_size<tuple<> > {
|
||||
public:
|
||||
static const size_t value = 0;
|
||||
};
|
||||
|
||||
template <>
|
||||
class tuple_size<null_type> {
|
||||
public:
|
||||
static const size_t value = 0;
|
||||
};
|
||||
|
||||
template<int __N, typename __T>
|
||||
struct tuple_element {
|
||||
typedef typename internal::component<__N, typename __T::my_cons>::type type;
|
||||
};
|
||||
|
||||
template<int __N, typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
|
||||
inline static typename tuple_element<__N,tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > >::type&
|
||||
get(tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK >& t) { return internal::get_helper<__N>::get(t); }
|
||||
|
||||
template<int __N, typename __T0, typename __T1, typename __T2, typename __T3, typename __T4 __TBB_TYPENAME_T_PACK >
|
||||
inline static typename tuple_element<__N,tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK > >::type const&
|
||||
get(const tuple<__T0,__T1,__T2,__T3,__T4 __TBB_T_PACK >& t) { return internal::get_helper<__N>::get(t); }
|
||||
|
||||
} // interface5
|
||||
} // tbb
|
||||
|
||||
#if !__TBB_CPP11_TUPLE_PRESENT
|
||||
namespace tbb {
|
||||
namespace flow {
|
||||
using tbb::interface5::tuple;
|
||||
using tbb::interface5::tuple_size;
|
||||
using tbb::interface5::tuple_element;
|
||||
using tbb::interface5::get;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
#undef __TBB_T_PACK
|
||||
#undef __TBB_U_PACK
|
||||
#undef __TBB_TYPENAME_T_PACK
|
||||
#undef __TBB_TYPENAME_U_PACK
|
||||
#undef __TBB_NULL_TYPE_PACK
|
||||
#undef __TBB_REF_T_PARAM_PACK
|
||||
#undef __TBB_CONST_REF_T_PARAM_PACK
|
||||
#undef __TBB_T_PARAM_LIST_PACK
|
||||
#undef __TBB_CONST_NULL_REF_PACK
|
||||
|
||||
#endif /* __TBB_tuple_H */
|
||||
@@ -0,0 +1,58 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/concurrent_hash_map.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
#if !TBB_NO_LEGACY
|
||||
struct hash_map_segment_base {
|
||||
typedef spin_rw_mutex segment_mutex_t;
|
||||
//! Type of a hash code.
|
||||
typedef size_t hashcode_t;
|
||||
//! Log2 of n_segment
|
||||
static const size_t n_segment_bits = 6;
|
||||
//! Maximum size of array of chains
|
||||
static const size_t max_physical_size = size_t(1)<<(8*sizeof(hashcode_t)-n_segment_bits);
|
||||
//! Mutex that protects this segment
|
||||
segment_mutex_t my_mutex;
|
||||
// Number of nodes
|
||||
atomic<size_t> my_logical_size;
|
||||
// Size of chains
|
||||
/** Always zero or a power of two */
|
||||
size_t my_physical_size;
|
||||
//! True if my_logical_size>=my_physical_size.
|
||||
/** Used to support Intel(R) Thread Checker. */
|
||||
bool __TBB_EXPORTED_METHOD internal_grow_predicate() const;
|
||||
};
|
||||
|
||||
bool hash_map_segment_base::internal_grow_predicate() const {
|
||||
// Intel(R) Thread Checker considers the following reads to be races, so we hide them in the
|
||||
// library so that Intel(R) Thread Checker will ignore them. The reads are used in a double-check
|
||||
// context, so the program is nonetheless correct despite the race.
|
||||
return my_logical_size >= my_physical_size && my_physical_size < max_physical_size;
|
||||
}
|
||||
#endif//!TBB_NO_LEGACY
|
||||
|
||||
} // namespace internal
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,235 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_concurrent_lru_cache_H
|
||||
#define __TBB_concurrent_lru_cache_H
|
||||
|
||||
#if ! TBB_PREVIEW_CONCURRENT_LRU_CACHE
|
||||
#error Set TBB_PREVIEW_CONCURRENT_LRU_CACHE to include concurrent_lru_cache.h
|
||||
#endif
|
||||
|
||||
#include <map>
|
||||
#include <list>
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
#include "atomic.h"
|
||||
#include "internal/_aggregator_impl.h"
|
||||
|
||||
namespace tbb{
|
||||
namespace interface6 {
|
||||
|
||||
|
||||
template <typename key_type, typename value_type, typename value_functor_type = value_type (*)(key_type) >
|
||||
class concurrent_lru_cache : internal::no_assign{
|
||||
private:
|
||||
typedef concurrent_lru_cache self_type;
|
||||
typedef value_functor_type value_function_type;
|
||||
typedef std::size_t ref_counter_type;
|
||||
struct map_value_type;
|
||||
typedef std::map<key_type, map_value_type> map_storage_type;
|
||||
typedef std::list<typename map_storage_type::iterator> lru_list_type;
|
||||
struct map_value_type {
|
||||
value_type my_value;
|
||||
ref_counter_type my_ref_counter;
|
||||
typename lru_list_type::iterator my_lru_list_iterator;
|
||||
bool my_is_ready;
|
||||
|
||||
map_value_type (value_type const& a_value, ref_counter_type a_ref_counter, typename lru_list_type::iterator a_lru_list_iterator, bool a_is_ready)
|
||||
: my_value(a_value), my_ref_counter(a_ref_counter), my_lru_list_iterator (a_lru_list_iterator), my_is_ready(a_is_ready)
|
||||
{}
|
||||
};
|
||||
|
||||
class handle_object;
|
||||
|
||||
struct aggregator_operation;
|
||||
typedef aggregator_operation aggregated_operation_type;
|
||||
typedef tbb::internal::aggregating_functor<self_type,aggregated_operation_type> aggregator_function_type;
|
||||
friend class tbb::internal::aggregating_functor<self_type,aggregated_operation_type>;
|
||||
typedef tbb::internal::aggregator<aggregator_function_type, aggregated_operation_type> aggregator_type;
|
||||
|
||||
private:
|
||||
value_function_type my_value_function;
|
||||
std::size_t const my_number_of_lru_history_items;
|
||||
map_storage_type my_map_storage;
|
||||
lru_list_type my_lru_list;
|
||||
aggregator_type my_aggregator;
|
||||
|
||||
public:
|
||||
typedef handle_object handle;
|
||||
|
||||
public:
|
||||
concurrent_lru_cache(value_function_type f, std::size_t number_of_lru_history_items)
|
||||
: my_value_function(f),my_number_of_lru_history_items(number_of_lru_history_items)
|
||||
{
|
||||
my_aggregator.initialize_handler(aggregator_function_type(this));
|
||||
}
|
||||
|
||||
handle_object operator[](key_type k){
|
||||
retrieve_aggregator_operation op(k);
|
||||
my_aggregator.execute(&op);
|
||||
if (op.is_new_value_needed()){
|
||||
op.result().second.my_value = my_value_function(k);
|
||||
__TBB_store_with_release(op.result().second.my_is_ready, true);
|
||||
}else{
|
||||
tbb::internal::spin_wait_while_eq(op.result().second.my_is_ready,false);
|
||||
}
|
||||
return handle_object(*this,op.result());
|
||||
}
|
||||
private:
|
||||
void signal_end_of_usage(typename map_storage_type::reference value_ref){
|
||||
signal_end_of_usage_aggregator_operation op(value_ref);
|
||||
my_aggregator.execute(&op);
|
||||
}
|
||||
|
||||
private:
|
||||
struct handle_move_t:no_assign{
|
||||
concurrent_lru_cache & my_cache_ref;
|
||||
typename map_storage_type::reference my_map_record_ref;
|
||||
handle_move_t(concurrent_lru_cache & cache_ref, typename map_storage_type::reference value_ref):my_cache_ref(cache_ref),my_map_record_ref(value_ref) {};
|
||||
};
|
||||
class handle_object {
|
||||
concurrent_lru_cache * my_cache_pointer;
|
||||
typename map_storage_type::reference my_map_record_ref;
|
||||
public:
|
||||
handle_object(concurrent_lru_cache & cache_ref, typename map_storage_type::reference value_ref):my_cache_pointer(&cache_ref), my_map_record_ref(value_ref) {}
|
||||
handle_object(handle_move_t m):my_cache_pointer(&m.my_cache_ref), my_map_record_ref(m.my_map_record_ref){}
|
||||
operator handle_move_t(){ return move(*this);}
|
||||
value_type& value(){
|
||||
__TBB_ASSERT(my_cache_pointer,"get value from moved from object?");
|
||||
return my_map_record_ref.second.my_value;
|
||||
}
|
||||
~handle_object(){
|
||||
if (my_cache_pointer){
|
||||
my_cache_pointer->signal_end_of_usage(my_map_record_ref);
|
||||
}
|
||||
}
|
||||
private:
|
||||
friend handle_move_t move(handle_object& h){
|
||||
return handle_object::move(h);
|
||||
}
|
||||
static handle_move_t move(handle_object& h){
|
||||
__TBB_ASSERT(h.my_cache_pointer,"move from the same object twice ?");
|
||||
concurrent_lru_cache * cache_pointer = NULL;
|
||||
std::swap(cache_pointer,h.my_cache_pointer);
|
||||
return handle_move_t(*cache_pointer,h.my_map_record_ref);
|
||||
}
|
||||
private:
|
||||
void operator=(handle_object&);
|
||||
#if __SUNPRO_CC
|
||||
// Presumably due to a compiler error, private copy constructor
|
||||
// breaks expressions like handle h = cache[key];
|
||||
public:
|
||||
#endif
|
||||
handle_object(handle_object &);
|
||||
};
|
||||
private:
|
||||
//TODO: looks like aggregator_operation is a perfect match for statically typed variant type
|
||||
struct aggregator_operation : tbb::internal::aggregated_operation<aggregator_operation>{
|
||||
enum e_op_type {op_retive, op_signal_end_of_usage};
|
||||
//TODO: try to use pointer to function apply_visitor here
|
||||
//TODO: try virtual functions and measure the difference
|
||||
e_op_type my_operation_type;
|
||||
aggregator_operation(e_op_type operation_type): my_operation_type(operation_type) {}
|
||||
void cast_and_handle(self_type& container ){
|
||||
if (my_operation_type==op_retive){
|
||||
static_cast<retrieve_aggregator_operation*>(this)->handle(container);
|
||||
}else{
|
||||
static_cast<signal_end_of_usage_aggregator_operation*>(this)->handle(container);
|
||||
}
|
||||
}
|
||||
};
|
||||
struct retrieve_aggregator_operation : aggregator_operation, private internal::no_assign {
|
||||
key_type my_key;
|
||||
typename map_storage_type::pointer my_result_map_record_pointer;
|
||||
bool my_is_new_value_needed;
|
||||
retrieve_aggregator_operation(key_type key):aggregator_operation(aggregator_operation::op_retive),my_key(key),my_is_new_value_needed(false){}
|
||||
void handle(self_type& container ){
|
||||
my_result_map_record_pointer = & container.retrieve_serial(my_key,my_is_new_value_needed);
|
||||
}
|
||||
typename map_storage_type::reference result(){ return * my_result_map_record_pointer; }
|
||||
bool is_new_value_needed(){return my_is_new_value_needed;}
|
||||
};
|
||||
struct signal_end_of_usage_aggregator_operation : aggregator_operation, private internal::no_assign {
|
||||
typename map_storage_type::reference my_map_record_ref;
|
||||
signal_end_of_usage_aggregator_operation(typename map_storage_type::reference map_record_ref):aggregator_operation(aggregator_operation::op_signal_end_of_usage),my_map_record_ref(map_record_ref){}
|
||||
void handle(self_type& container ){
|
||||
container.signal_end_of_usage_serial(my_map_record_ref);
|
||||
}
|
||||
};
|
||||
|
||||
private:
|
||||
void handle_operations(aggregator_operation* op_list){
|
||||
while(op_list){
|
||||
op_list->cast_and_handle(*this);
|
||||
aggregator_operation* tmp = op_list;
|
||||
op_list=op_list->next;
|
||||
tbb::internal::itt_store_word_with_release(tmp->status, uintptr_t(1));
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
typename map_storage_type::reference retrieve_serial(key_type k, bool& is_new_value_needed){
|
||||
typename map_storage_type::iterator it = my_map_storage.find(k);
|
||||
if (it == my_map_storage.end()){
|
||||
it = my_map_storage.insert(it,std::make_pair(k,map_value_type(value_type(),0,my_lru_list.end(),false)));
|
||||
is_new_value_needed = true;
|
||||
}else {
|
||||
typename lru_list_type::iterator list_it = it->second.my_lru_list_iterator;
|
||||
if (list_it!=my_lru_list.end()) {
|
||||
__TBB_ASSERT(!it->second.my_ref_counter,"item to be evicted should not have a live references");
|
||||
//item is going to be used. Therefore it is not a subject for eviction
|
||||
//so - remove it from LRU history.
|
||||
my_lru_list.erase(list_it);
|
||||
it->second.my_lru_list_iterator= my_lru_list.end();
|
||||
}
|
||||
}
|
||||
++(it->second.my_ref_counter);
|
||||
return *it;
|
||||
}
|
||||
|
||||
void signal_end_of_usage_serial(typename map_storage_type::reference map_record_ref){
|
||||
typename map_storage_type::iterator it = my_map_storage.find(map_record_ref.first);
|
||||
__TBB_ASSERT(it!=my_map_storage.end(),"cache should not return past-end iterators to outer world");
|
||||
__TBB_ASSERT(&(*it) == &map_record_ref,"dangling reference has been returned to outside world? data race ?");
|
||||
__TBB_ASSERT( my_lru_list.end()== std::find(my_lru_list.begin(),my_lru_list.end(),it),
|
||||
"object in use should not be in list of unused objects ");
|
||||
if (! --(it->second.my_ref_counter)){
|
||||
//it was the last reference so put it to the LRU history
|
||||
if (my_lru_list.size()>=my_number_of_lru_history_items){
|
||||
//evict items in order to get a space
|
||||
size_t number_of_elements_to_evict = 1 + my_lru_list.size() - my_number_of_lru_history_items;
|
||||
for (size_t i=0; i<number_of_elements_to_evict; ++i){
|
||||
typename map_storage_type::iterator it_to_evict = my_lru_list.back();
|
||||
__TBB_ASSERT(!it_to_evict->second.my_ref_counter,"item to be evicted should not have a live references");
|
||||
my_lru_list.pop_back();
|
||||
my_map_storage.erase(it_to_evict);
|
||||
}
|
||||
}
|
||||
my_lru_list.push_front(it);
|
||||
it->second.my_lru_list_iterator = my_lru_list.begin();
|
||||
}
|
||||
}
|
||||
};
|
||||
} // namespace interface6
|
||||
|
||||
using interface6::concurrent_lru_cache;
|
||||
|
||||
} // namespace tbb
|
||||
#endif //__TBB_concurrent_lru_cache_H
|
||||
@@ -0,0 +1,137 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "concurrent_monitor.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
void concurrent_monitor::thread_context::init() {
|
||||
new (sema.begin()) binary_semaphore;
|
||||
ready = true;
|
||||
}
|
||||
|
||||
concurrent_monitor::~concurrent_monitor() {
|
||||
abort_all();
|
||||
__TBB_ASSERT( waitset_ec.empty(), "waitset not empty?" );
|
||||
}
|
||||
|
||||
void concurrent_monitor::prepare_wait( thread_context& thr, uintptr_t ctx ) {
|
||||
if( !thr.ready )
|
||||
thr.init();
|
||||
// this is good place to pump previous spurious wakeup
|
||||
else if( thr.spurious ) {
|
||||
thr.spurious = false;
|
||||
thr.semaphore().P();
|
||||
}
|
||||
thr.context = ctx;
|
||||
thr.in_waitset = true;
|
||||
{
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
__TBB_store_relaxed( thr.epoch, __TBB_load_relaxed(epoch) );
|
||||
waitset_ec.add( (waitset_t::node_t*)&thr );
|
||||
}
|
||||
atomic_fence();
|
||||
}
|
||||
|
||||
void concurrent_monitor::cancel_wait( thread_context& thr ) {
|
||||
// spurious wakeup will be pumped in the following prepare_wait()
|
||||
thr.spurious = true;
|
||||
// try to remove node from waitset
|
||||
bool th_in_waitset = thr.in_waitset;
|
||||
if( th_in_waitset ) {
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
if (thr.in_waitset) {
|
||||
// successfully removed from waitset,
|
||||
// so there will be no spurious wakeup
|
||||
thr.in_waitset = false;
|
||||
thr.spurious = false;
|
||||
waitset_ec.remove( (waitset_t::node_t&)thr );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void concurrent_monitor::notify_one_relaxed() {
|
||||
if( waitset_ec.empty() )
|
||||
return;
|
||||
waitset_node_t* n;
|
||||
const waitset_node_t* end = waitset_ec.end();
|
||||
{
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
|
||||
n = waitset_ec.front();
|
||||
if( n!=end ) {
|
||||
waitset_ec.remove( *n );
|
||||
to_thread_context(n)->in_waitset = false;
|
||||
}
|
||||
}
|
||||
if( n!=end )
|
||||
to_thread_context(n)->semaphore().V();
|
||||
}
|
||||
|
||||
void concurrent_monitor::notify_all_relaxed() {
|
||||
if( waitset_ec.empty() )
|
||||
return;
|
||||
dllist_t temp;
|
||||
const waitset_node_t* end;
|
||||
{
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
|
||||
waitset_ec.flush_to( temp );
|
||||
end = temp.end();
|
||||
for( waitset_node_t* n=temp.front(); n!=end; n=n->next )
|
||||
to_thread_context(n)->in_waitset = false;
|
||||
}
|
||||
waitset_node_t* nxt;
|
||||
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
|
||||
nxt = n->next;
|
||||
to_thread_context(n)->semaphore().V();
|
||||
}
|
||||
#if TBB_USE_ASSERT
|
||||
temp.clear();
|
||||
#endif
|
||||
}
|
||||
|
||||
void concurrent_monitor::abort_all_relaxed() {
|
||||
if( waitset_ec.empty() )
|
||||
return;
|
||||
dllist_t temp;
|
||||
const waitset_node_t* end;
|
||||
{
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
__TBB_store_relaxed( epoch, __TBB_load_relaxed(epoch) + 1 );
|
||||
waitset_ec.flush_to( temp );
|
||||
end = temp.end();
|
||||
for( waitset_node_t* n=temp.front(); n!=end; n=n->next )
|
||||
to_thread_context(n)->in_waitset = false;
|
||||
}
|
||||
waitset_node_t* nxt;
|
||||
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
|
||||
nxt = n->next;
|
||||
to_thread_context(n)->aborted = true;
|
||||
to_thread_context(n)->semaphore().V();
|
||||
}
|
||||
#if TBB_USE_ASSERT
|
||||
temp.clear();
|
||||
#endif
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
@@ -0,0 +1,241 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_concurrent_monitor_H
|
||||
#define __TBB_concurrent_monitor_H
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
#include "tbb/atomic.h"
|
||||
#include "tbb/spin_mutex.h"
|
||||
#include "tbb/tbb_exception.h"
|
||||
#include "tbb/aligned_space.h"
|
||||
|
||||
#include "semaphore.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
//! Circular doubly-linked list with sentinel
|
||||
/** head.next points to the front and head.prev points to the back */
|
||||
class circular_doubly_linked_list_with_sentinel : no_copy {
|
||||
public:
|
||||
struct node_t {
|
||||
node_t* next;
|
||||
node_t* prev;
|
||||
explicit node_t() : next((node_t*)(uintptr_t)0xcdcdcdcd), prev((node_t*)(uintptr_t)0xcdcdcdcd) {}
|
||||
};
|
||||
|
||||
// ctor
|
||||
circular_doubly_linked_list_with_sentinel() {clear();}
|
||||
// dtor
|
||||
~circular_doubly_linked_list_with_sentinel() {__TBB_ASSERT( head.next==&head && head.prev==&head, "the list is not empty" );}
|
||||
|
||||
inline size_t size() const {return count;}
|
||||
inline bool empty() const {return size()==0;}
|
||||
inline node_t* front() const {return head.next;}
|
||||
inline node_t* last() const {return head.prev;}
|
||||
inline node_t* begin() const {return front();}
|
||||
inline const node_t* end() const {return &head;}
|
||||
|
||||
//! add to the back of the list
|
||||
inline void add( node_t* n ) {
|
||||
__TBB_store_relaxed(count, __TBB_load_relaxed(count) + 1);
|
||||
n->prev = head.prev;
|
||||
n->next = &head;
|
||||
head.prev->next = n;
|
||||
head.prev = n;
|
||||
}
|
||||
|
||||
//! remove node 'n'
|
||||
inline void remove( node_t& n ) {
|
||||
__TBB_store_relaxed(count, __TBB_load_relaxed(count) - 1);
|
||||
n.prev->next = n.next;
|
||||
n.next->prev = n.prev;
|
||||
}
|
||||
|
||||
//! move all elements to 'lst' and initialize the 'this' list
|
||||
inline void flush_to( circular_doubly_linked_list_with_sentinel& lst ) {
|
||||
if( const size_t l_count = __TBB_load_relaxed(count) ) {
|
||||
__TBB_store_relaxed(lst.count, l_count);
|
||||
lst.head.next = head.next;
|
||||
lst.head.prev = head.prev;
|
||||
head.next->prev = &lst.head;
|
||||
head.prev->next = &lst.head;
|
||||
clear();
|
||||
}
|
||||
}
|
||||
|
||||
void clear() {head.next = head.prev = &head; __TBB_store_relaxed(count, 0);}
|
||||
private:
|
||||
__TBB_atomic size_t count;
|
||||
node_t head;
|
||||
};
|
||||
|
||||
typedef circular_doubly_linked_list_with_sentinel waitset_t;
|
||||
typedef circular_doubly_linked_list_with_sentinel dllist_t;
|
||||
typedef circular_doubly_linked_list_with_sentinel::node_t waitset_node_t;
|
||||
|
||||
//! concurrent_monitor
|
||||
/** fine-grained concurrent_monitor implementation */
|
||||
class concurrent_monitor : no_copy {
|
||||
public:
|
||||
/** per-thread descriptor for concurrent_monitor */
|
||||
class thread_context : waitset_node_t, no_copy {
|
||||
friend class concurrent_monitor;
|
||||
public:
|
||||
thread_context() : spurious(false), aborted(false), ready(false), context(0) {
|
||||
epoch = 0;
|
||||
in_waitset = false;
|
||||
}
|
||||
~thread_context() {
|
||||
if (ready) {
|
||||
if( spurious ) semaphore().P();
|
||||
semaphore().~binary_semaphore();
|
||||
}
|
||||
}
|
||||
binary_semaphore& semaphore() { return *sema.begin(); }
|
||||
private:
|
||||
//! The method for lazy initialization of the thread_context's semaphore.
|
||||
// Inlining of the method is undesirable, due to extra instructions for
|
||||
// exception support added at caller side.
|
||||
__TBB_NOINLINE( void init() );
|
||||
tbb::aligned_space<binary_semaphore> sema;
|
||||
__TBB_atomic unsigned epoch;
|
||||
tbb::atomic<bool> in_waitset;
|
||||
bool spurious;
|
||||
bool aborted;
|
||||
bool ready;
|
||||
uintptr_t context;
|
||||
};
|
||||
|
||||
//! ctor
|
||||
concurrent_monitor() {__TBB_store_relaxed(epoch, 0);}
|
||||
|
||||
//! dtor
|
||||
~concurrent_monitor() ;
|
||||
|
||||
//! prepare wait by inserting 'thr' into the wait queue
|
||||
void prepare_wait( thread_context& thr, uintptr_t ctx = 0 );
|
||||
|
||||
//! Commit wait if event count has not changed; otherwise, cancel wait.
|
||||
/** Returns true if committed, false if canceled. */
|
||||
inline bool commit_wait( thread_context& thr ) {
|
||||
const bool do_it = thr.epoch == __TBB_load_relaxed(epoch);
|
||||
// this check is just an optimization
|
||||
if( do_it ) {
|
||||
__TBB_ASSERT( thr.ready, "use of commit_wait() without prior prepare_wait()");
|
||||
thr.semaphore().P();
|
||||
__TBB_ASSERT( !thr.in_waitset, "still in the queue?" );
|
||||
if( thr.aborted )
|
||||
throw_exception( eid_user_abort );
|
||||
} else {
|
||||
cancel_wait( thr );
|
||||
}
|
||||
return do_it;
|
||||
}
|
||||
//! Cancel the wait. Removes the thread from the wait queue if not removed yet.
|
||||
void cancel_wait( thread_context& thr );
|
||||
|
||||
//! Wait for a condition to be satisfied with waiting-on context
|
||||
template<typename WaitUntil, typename Context>
|
||||
void wait( WaitUntil until, Context on );
|
||||
|
||||
//! Notify one thread about the event
|
||||
void notify_one() {atomic_fence(); notify_one_relaxed();}
|
||||
|
||||
//! Notify one thread about the event. Relaxed version.
|
||||
void notify_one_relaxed();
|
||||
|
||||
//! Notify all waiting threads of the event
|
||||
void notify_all() {atomic_fence(); notify_all_relaxed();}
|
||||
|
||||
//! Notify all waiting threads of the event; Relaxed version
|
||||
void notify_all_relaxed();
|
||||
|
||||
//! Notify waiting threads of the event that satisfies the given predicate
|
||||
template<typename P> void notify( const P& predicate ) {atomic_fence(); notify_relaxed( predicate );}
|
||||
|
||||
//! Notify waiting threads of the event that satisfies the given predicate; Relaxed version
|
||||
template<typename P> void notify_relaxed( const P& predicate );
|
||||
|
||||
//! Abort any sleeping threads at the time of the call
|
||||
void abort_all() {atomic_fence(); abort_all_relaxed(); }
|
||||
|
||||
//! Abort any sleeping threads at the time of the call; Relaxed version
|
||||
void abort_all_relaxed();
|
||||
|
||||
private:
|
||||
tbb::spin_mutex mutex_ec;
|
||||
waitset_t waitset_ec;
|
||||
__TBB_atomic unsigned epoch;
|
||||
thread_context* to_thread_context( waitset_node_t* n ) { return static_cast<thread_context*>(n); }
|
||||
};
|
||||
|
||||
template<typename WaitUntil, typename Context>
|
||||
void concurrent_monitor::wait( WaitUntil until, Context on )
|
||||
{
|
||||
bool slept = false;
|
||||
thread_context thr_ctx;
|
||||
prepare_wait( thr_ctx, on() );
|
||||
while( !until() ) {
|
||||
if( (slept = commit_wait( thr_ctx ) )==true )
|
||||
if( until() ) break;
|
||||
slept = false;
|
||||
prepare_wait( thr_ctx, on() );
|
||||
}
|
||||
if( !slept )
|
||||
cancel_wait( thr_ctx );
|
||||
}
|
||||
|
||||
template<typename P>
|
||||
void concurrent_monitor::notify_relaxed( const P& predicate ) {
|
||||
if( waitset_ec.empty() )
|
||||
return;
|
||||
dllist_t temp;
|
||||
waitset_node_t* nxt;
|
||||
const waitset_node_t* end = waitset_ec.end();
|
||||
{
|
||||
tbb::spin_mutex::scoped_lock l( mutex_ec );
|
||||
__TBB_store_relaxed(epoch, __TBB_load_relaxed(epoch) + 1);
|
||||
for( waitset_node_t* n=waitset_ec.last(); n!=end; n=nxt ) {
|
||||
nxt = n->prev;
|
||||
thread_context* thr = to_thread_context( n );
|
||||
if( predicate( thr->context ) ) {
|
||||
waitset_ec.remove( *n );
|
||||
thr->in_waitset = false;
|
||||
temp.add( n );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
end = temp.end();
|
||||
for( waitset_node_t* n=temp.front(); n!=end; n=nxt ) {
|
||||
nxt = n->next;
|
||||
to_thread_context(n)->semaphore().V();
|
||||
}
|
||||
#if TBB_USE_ASSERT
|
||||
temp.clear();
|
||||
#endif
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_concurrent_monitor_H */
|
||||
@@ -0,0 +1,457 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_concurrent_priority_queue_H
|
||||
#define __TBB_concurrent_priority_queue_H
|
||||
|
||||
#include "atomic.h"
|
||||
#include "cache_aligned_allocator.h"
|
||||
#include "tbb_exception.h"
|
||||
#include "tbb_stddef.h"
|
||||
#include "tbb_profiling.h"
|
||||
#include "internal/_aggregator_impl.h"
|
||||
#include <vector>
|
||||
#include <iterator>
|
||||
#include <functional>
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
#include <initializer_list>
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
namespace interface5 {
|
||||
|
||||
using namespace tbb::internal;
|
||||
|
||||
//! Concurrent priority queue
|
||||
template <typename T, typename Compare=std::less<T>, typename A=cache_aligned_allocator<T> >
|
||||
class concurrent_priority_queue {
|
||||
public:
|
||||
//! Element type in the queue.
|
||||
typedef T value_type;
|
||||
|
||||
//! Reference type
|
||||
typedef T& reference;
|
||||
|
||||
//! Const reference type
|
||||
typedef const T& const_reference;
|
||||
|
||||
//! Integral type for representing size of the queue.
|
||||
typedef size_t size_type;
|
||||
|
||||
//! Difference type for iterator
|
||||
typedef ptrdiff_t difference_type;
|
||||
|
||||
//! Allocator type
|
||||
typedef A allocator_type;
|
||||
|
||||
//! Constructs a new concurrent_priority_queue with default capacity
|
||||
explicit concurrent_priority_queue(const allocator_type& a = allocator_type()) : mark(0), my_size(0), data(a)
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
}
|
||||
|
||||
//! Constructs a new concurrent_priority_queue with init_sz capacity
|
||||
explicit concurrent_priority_queue(size_type init_capacity, const allocator_type& a = allocator_type()) :
|
||||
mark(0), my_size(0), data(a)
|
||||
{
|
||||
data.reserve(init_capacity);
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
}
|
||||
|
||||
//! [begin,end) constructor
|
||||
template<typename InputIterator>
|
||||
concurrent_priority_queue(InputIterator begin, InputIterator end, const allocator_type& a = allocator_type()) :
|
||||
mark(0), data(begin, end, a)
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
heapify();
|
||||
my_size = data.size();
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Constructor from std::initializer_list
|
||||
concurrent_priority_queue(std::initializer_list<T> init_list, const allocator_type &a = allocator_type()) :
|
||||
mark(0),data(init_list.begin(), init_list.end(), a)
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
heapify();
|
||||
my_size = data.size();
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
//! Copy constructor
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
explicit concurrent_priority_queue(const concurrent_priority_queue& src) : mark(src.mark),
|
||||
my_size(src.my_size), data(src.data.begin(), src.data.end(), src.data.get_allocator())
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
heapify();
|
||||
}
|
||||
|
||||
//! Copy constructor with specific allocator
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
concurrent_priority_queue(const concurrent_priority_queue& src, const allocator_type& a) : mark(src.mark),
|
||||
my_size(src.my_size), data(src.data.begin(), src.data.end(), a)
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
heapify();
|
||||
}
|
||||
|
||||
//! Assignment operator
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
concurrent_priority_queue& operator=(const concurrent_priority_queue& src) {
|
||||
if (this != &src) {
|
||||
vector_t(src.data.begin(), src.data.end(), src.data.get_allocator()).swap(data);
|
||||
mark = src.mark;
|
||||
my_size = src.my_size;
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Move constructor
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
concurrent_priority_queue(concurrent_priority_queue&& src) : mark(src.mark),
|
||||
my_size(src.my_size), data(std::move(src.data))
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
}
|
||||
|
||||
//! Move constructor with specific allocator
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
concurrent_priority_queue(concurrent_priority_queue&& src, const allocator_type& a) : mark(src.mark),
|
||||
my_size(src.my_size),
|
||||
#if __TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
data(std::move(src.data), a)
|
||||
#else
|
||||
// Some early version of C++11 STL vector does not have a constructor of vector(vector&& , allocator).
|
||||
// It seems that the reason is absence of support of allocator_traits (stateful allocators).
|
||||
data(a)
|
||||
#endif //__TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
{
|
||||
my_aggregator.initialize_handler(my_functor_t(this));
|
||||
#if !__TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
if (a != src.data.get_allocator()){
|
||||
data.reserve(src.data.size());
|
||||
data.assign(std::make_move_iterator(src.data.begin()), std::make_move_iterator(src.data.end()));
|
||||
}else{
|
||||
data = std::move(src.data);
|
||||
}
|
||||
#endif //!__TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
}
|
||||
|
||||
//! Move assignment operator
|
||||
/** This operation is unsafe if there are pending concurrent operations on the src queue. */
|
||||
concurrent_priority_queue& operator=( concurrent_priority_queue&& src) {
|
||||
if (this != &src) {
|
||||
mark = src.mark;
|
||||
my_size = src.my_size;
|
||||
#if !__TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
if (data.get_allocator() != src.data.get_allocator()){
|
||||
vector_t(std::make_move_iterator(src.data.begin()), std::make_move_iterator(src.data.end()), data.get_allocator()).swap(data);
|
||||
}else
|
||||
#endif //!__TBB_ALLOCATOR_TRAITS_PRESENT
|
||||
{
|
||||
data = std::move(src.data);
|
||||
}
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
|
||||
|
||||
//! Assign the queue from [begin,end) range, not thread-safe
|
||||
template<typename InputIterator>
|
||||
void assign(InputIterator begin, InputIterator end) {
|
||||
vector_t(begin, end, data.get_allocator()).swap(data);
|
||||
mark = 0;
|
||||
my_size = data.size();
|
||||
heapify();
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Assign the queue from std::initializer_list, not thread-safe
|
||||
void assign(std::initializer_list<T> il) { this->assign(il.begin(), il.end()); }
|
||||
|
||||
//! Assign from std::initializer_list, not thread-safe
|
||||
concurrent_priority_queue& operator=(std::initializer_list<T> il) {
|
||||
this->assign(il.begin(), il.end());
|
||||
return *this;
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
//! Returns true if empty, false otherwise
|
||||
/** Returned value may not reflect results of pending operations.
|
||||
This operation reads shared data and will trigger a race condition. */
|
||||
bool empty() const { return size()==0; }
|
||||
|
||||
//! Returns the current number of elements contained in the queue
|
||||
/** Returned value may not reflect results of pending operations.
|
||||
This operation reads shared data and will trigger a race condition. */
|
||||
size_type size() const { return __TBB_load_with_acquire(my_size); }
|
||||
|
||||
//! Pushes elem onto the queue, increasing capacity of queue if necessary
|
||||
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
|
||||
void push(const_reference elem) {
|
||||
cpq_operation op_data(elem, PUSH_OP);
|
||||
my_aggregator.execute(&op_data);
|
||||
if (op_data.status == FAILED) // exception thrown
|
||||
throw_exception(eid_bad_alloc);
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Pushes elem onto the queue, increasing capacity of queue if necessary
|
||||
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
|
||||
void push(value_type &&elem) {
|
||||
cpq_operation op_data(elem, PUSH_RVALUE_OP);
|
||||
my_aggregator.execute(&op_data);
|
||||
if (op_data.status == FAILED) // exception thrown
|
||||
throw_exception(eid_bad_alloc);
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
|
||||
//! Constructs a new element using args as the arguments for its construction and pushes it onto the queue */
|
||||
/** This operation can be safely used concurrently with other push, try_pop or emplace operations. */
|
||||
template<typename... Args>
|
||||
void emplace(Args&&... args) {
|
||||
push(value_type(std::forward<Args>(args)...));
|
||||
}
|
||||
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! Gets a reference to and removes highest priority element
|
||||
/** If a highest priority element was found, sets elem and returns true,
|
||||
otherwise returns false.
|
||||
This operation can be safely used concurrently with other push, try_pop or emplace operations. */
|
||||
bool try_pop(reference elem) {
|
||||
cpq_operation op_data(POP_OP);
|
||||
op_data.elem = &elem;
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.status==SUCCEEDED;
|
||||
}
|
||||
|
||||
//! Clear the queue; not thread-safe
|
||||
/** This operation is unsafe if there are pending concurrent operations on the queue.
|
||||
Resets size, effectively emptying queue; does not free space.
|
||||
May not clear elements added in pending operations. */
|
||||
void clear() {
|
||||
data.clear();
|
||||
mark = 0;
|
||||
my_size = 0;
|
||||
}
|
||||
|
||||
//! Swap this queue with another; not thread-safe
|
||||
/** This operation is unsafe if there are pending concurrent operations on the queue. */
|
||||
void swap(concurrent_priority_queue& q) {
|
||||
using std::swap;
|
||||
data.swap(q.data);
|
||||
swap(mark, q.mark);
|
||||
swap(my_size, q.my_size);
|
||||
}
|
||||
|
||||
//! Return allocator object
|
||||
allocator_type get_allocator() const { return data.get_allocator(); }
|
||||
|
||||
private:
|
||||
enum operation_type {INVALID_OP, PUSH_OP, POP_OP, PUSH_RVALUE_OP};
|
||||
enum operation_status { WAIT=0, SUCCEEDED, FAILED };
|
||||
|
||||
class cpq_operation : public aggregated_operation<cpq_operation> {
|
||||
public:
|
||||
operation_type type;
|
||||
union {
|
||||
value_type *elem;
|
||||
size_type sz;
|
||||
};
|
||||
cpq_operation(const_reference e, operation_type t) :
|
||||
type(t), elem(const_cast<value_type*>(&e)) {}
|
||||
cpq_operation(operation_type t) : type(t) {}
|
||||
};
|
||||
|
||||
class my_functor_t {
|
||||
concurrent_priority_queue<T, Compare, A> *cpq;
|
||||
public:
|
||||
my_functor_t() {}
|
||||
my_functor_t(concurrent_priority_queue<T, Compare, A> *cpq_) : cpq(cpq_) {}
|
||||
void operator()(cpq_operation* op_list) {
|
||||
cpq->handle_operations(op_list);
|
||||
}
|
||||
};
|
||||
|
||||
typedef tbb::internal::aggregator< my_functor_t, cpq_operation > aggregator_t;
|
||||
aggregator_t my_aggregator;
|
||||
//! Padding added to avoid false sharing
|
||||
char padding1[NFS_MaxLineSize - sizeof(aggregator_t)];
|
||||
//! The point at which unsorted elements begin
|
||||
size_type mark;
|
||||
__TBB_atomic size_type my_size;
|
||||
Compare compare;
|
||||
//! Padding added to avoid false sharing
|
||||
char padding2[NFS_MaxLineSize - (2*sizeof(size_type)) - sizeof(Compare)];
|
||||
//! Storage for the heap of elements in queue, plus unheapified elements
|
||||
/** data has the following structure:
|
||||
|
||||
binary unheapified
|
||||
heap elements
|
||||
____|_______|____
|
||||
| | |
|
||||
v v v
|
||||
[_|...|_|_|...|_| |...| ]
|
||||
0 ^ ^ ^
|
||||
| | |__capacity
|
||||
| |__my_size
|
||||
|__mark
|
||||
|
||||
Thus, data stores the binary heap starting at position 0 through
|
||||
mark-1 (it may be empty). Then there are 0 or more elements
|
||||
that have not yet been inserted into the heap, in positions
|
||||
mark through my_size-1. */
|
||||
typedef std::vector<value_type, allocator_type> vector_t;
|
||||
vector_t data;
|
||||
|
||||
void handle_operations(cpq_operation *op_list) {
|
||||
cpq_operation *tmp, *pop_list=NULL;
|
||||
|
||||
__TBB_ASSERT(mark == data.size(), NULL);
|
||||
|
||||
// First pass processes all constant (amortized; reallocation may happen) time pushes and pops.
|
||||
while (op_list) {
|
||||
// ITT note: &(op_list->status) tag is used to cover accesses to op_list
|
||||
// node. This thread is going to handle the operation, and so will acquire it
|
||||
// and perform the associated operation w/o triggering a race condition; the
|
||||
// thread that created the operation is waiting on the status field, so when
|
||||
// this thread is done with the operation, it will perform a
|
||||
// store_with_release to give control back to the waiting thread in
|
||||
// aggregator::insert_operation.
|
||||
call_itt_notify(acquired, &(op_list->status));
|
||||
__TBB_ASSERT(op_list->type != INVALID_OP, NULL);
|
||||
tmp = op_list;
|
||||
op_list = itt_hide_load_word(op_list->next);
|
||||
if (tmp->type == POP_OP) {
|
||||
if (mark < data.size() &&
|
||||
compare(data[0], data[data.size()-1])) {
|
||||
// there are newly pushed elems and the last one
|
||||
// is higher than top
|
||||
*(tmp->elem) = move(data[data.size()-1]);
|
||||
__TBB_store_with_release(my_size, my_size-1);
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
|
||||
data.pop_back();
|
||||
__TBB_ASSERT(mark<=data.size(), NULL);
|
||||
}
|
||||
else { // no convenient item to pop; postpone
|
||||
itt_hide_store_word(tmp->next, pop_list);
|
||||
pop_list = tmp;
|
||||
}
|
||||
} else { // PUSH_OP or PUSH_RVALUE_OP
|
||||
__TBB_ASSERT(tmp->type == PUSH_OP || tmp->type == PUSH_RVALUE_OP, "Unknown operation" );
|
||||
__TBB_TRY{
|
||||
if (tmp->type == PUSH_OP) {
|
||||
data.push_back(*(tmp->elem));
|
||||
} else {
|
||||
data.push_back(move(*(tmp->elem)));
|
||||
}
|
||||
__TBB_store_with_release(my_size, my_size + 1);
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
|
||||
} __TBB_CATCH(...) {
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(FAILED));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// second pass processes pop operations
|
||||
while (pop_list) {
|
||||
tmp = pop_list;
|
||||
pop_list = itt_hide_load_word(pop_list->next);
|
||||
__TBB_ASSERT(tmp->type == POP_OP, NULL);
|
||||
if (data.empty()) {
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(FAILED));
|
||||
}
|
||||
else {
|
||||
__TBB_ASSERT(mark<=data.size(), NULL);
|
||||
if (mark < data.size() &&
|
||||
compare(data[0], data[data.size()-1])) {
|
||||
// there are newly pushed elems and the last one is
|
||||
// higher than top
|
||||
*(tmp->elem) = move(data[data.size()-1]);
|
||||
__TBB_store_with_release(my_size, my_size-1);
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
|
||||
data.pop_back();
|
||||
}
|
||||
else { // extract top and push last element down heap
|
||||
*(tmp->elem) = move(data[0]);
|
||||
__TBB_store_with_release(my_size, my_size-1);
|
||||
itt_store_word_with_release(tmp->status, uintptr_t(SUCCEEDED));
|
||||
reheap();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// heapify any leftover pushed elements before doing the next
|
||||
// batch of operations
|
||||
if (mark<data.size()) heapify();
|
||||
__TBB_ASSERT(mark == data.size(), NULL);
|
||||
}
|
||||
|
||||
//! Merge unsorted elements into heap
|
||||
void heapify() {
|
||||
if (!mark && data.size()>0) mark = 1;
|
||||
for (; mark<data.size(); ++mark) {
|
||||
// for each unheapified element under size
|
||||
size_type cur_pos = mark;
|
||||
value_type to_place = move(data[mark]);
|
||||
do { // push to_place up the heap
|
||||
size_type parent = (cur_pos-1)>>1;
|
||||
if (!compare(data[parent], to_place)) break;
|
||||
data[cur_pos] = move(data[parent]);
|
||||
cur_pos = parent;
|
||||
} while( cur_pos );
|
||||
data[cur_pos] = move(to_place);
|
||||
}
|
||||
}
|
||||
|
||||
//! Re-heapify after an extraction
|
||||
/** Re-heapify by pushing last element down the heap from the root. */
|
||||
void reheap() {
|
||||
size_type cur_pos=0, child=1;
|
||||
|
||||
while (child < mark) {
|
||||
size_type target = child;
|
||||
if (child+1 < mark && compare(data[child], data[child+1]))
|
||||
++target;
|
||||
// target now has the higher priority child
|
||||
if (compare(data[target], data[data.size()-1])) break;
|
||||
data[cur_pos] = move(data[target]);
|
||||
cur_pos = target;
|
||||
child = (cur_pos<<1)+1;
|
||||
}
|
||||
if (cur_pos != data.size()-1)
|
||||
data[cur_pos] = move(data[data.size()-1]);
|
||||
data.pop_back();
|
||||
if (mark > data.size()) mark = data.size();
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace interface5
|
||||
|
||||
using interface5::concurrent_priority_queue;
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_concurrent_priority_queue_H */
|
||||
@@ -0,0 +1,670 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
#include "tbb/tbb_machine.h"
|
||||
#include "tbb/tbb_exception.h"
|
||||
// Define required to satisfy test in internal file.
|
||||
#define __TBB_concurrent_queue_H
|
||||
#include "tbb/internal/_concurrent_queue_impl.h"
|
||||
#include "concurrent_monitor.h"
|
||||
#include "itt_notify.h"
|
||||
#include <new>
|
||||
|
||||
#if !TBB_USE_EXCEPTIONS && _MSC_VER
|
||||
// Suppress "C++ exception handler used, but unwind semantics are not enabled" warning in STL headers
|
||||
#pragma warning (push)
|
||||
#pragma warning (disable: 4530)
|
||||
#endif
|
||||
|
||||
#include <cstring> // for memset()
|
||||
|
||||
#if !TBB_USE_EXCEPTIONS && _MSC_VER
|
||||
#pragma warning (pop)
|
||||
#endif
|
||||
|
||||
using namespace std;
|
||||
|
||||
#if defined(_MSC_VER) && defined(_Wp64)
|
||||
// Workaround for overzealous compiler warnings in /Wp64 mode
|
||||
#pragma warning (disable: 4267)
|
||||
#endif
|
||||
|
||||
#define RECORD_EVENTS 0
|
||||
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
|
||||
typedef concurrent_queue_base_v3 concurrent_queue_base;
|
||||
|
||||
typedef size_t ticket;
|
||||
|
||||
//! A queue using simple locking.
|
||||
/** For efficiency, this class has no constructor.
|
||||
The caller is expected to zero-initialize it. */
|
||||
struct micro_queue {
|
||||
typedef concurrent_queue_base::page page;
|
||||
|
||||
friend class micro_queue_pop_finalizer;
|
||||
|
||||
atomic<page*> head_page;
|
||||
atomic<ticket> head_counter;
|
||||
|
||||
atomic<page*> tail_page;
|
||||
atomic<ticket> tail_counter;
|
||||
|
||||
spin_mutex page_mutex;
|
||||
|
||||
void push( const void* item, ticket k, concurrent_queue_base& base,
|
||||
concurrent_queue_base::copy_specifics op_type );
|
||||
|
||||
void abort_push( ticket k, concurrent_queue_base& base );
|
||||
|
||||
bool pop( void* dst, ticket k, concurrent_queue_base& base );
|
||||
|
||||
micro_queue& assign( const micro_queue& src, concurrent_queue_base& base,
|
||||
concurrent_queue_base::copy_specifics op_type );
|
||||
|
||||
page* make_copy ( concurrent_queue_base& base, const page* src_page, size_t begin_in_page,
|
||||
size_t end_in_page, ticket& g_index, concurrent_queue_base::copy_specifics op_type ) ;
|
||||
|
||||
void make_invalid( ticket k );
|
||||
};
|
||||
|
||||
// we need to yank it out of micro_queue because of concurrent_queue_base::deallocate_page being virtual.
|
||||
class micro_queue_pop_finalizer: no_copy {
|
||||
typedef concurrent_queue_base::page page;
|
||||
ticket my_ticket;
|
||||
micro_queue& my_queue;
|
||||
page* my_page;
|
||||
concurrent_queue_base &base;
|
||||
public:
|
||||
micro_queue_pop_finalizer( micro_queue& queue, concurrent_queue_base& b, ticket k, page* p ) :
|
||||
my_ticket(k), my_queue(queue), my_page(p), base(b)
|
||||
{}
|
||||
~micro_queue_pop_finalizer() {
|
||||
page* p = my_page;
|
||||
if( p ) {
|
||||
spin_mutex::scoped_lock lock( my_queue.page_mutex );
|
||||
page* q = p->next;
|
||||
my_queue.head_page = q;
|
||||
if( !q ) {
|
||||
my_queue.tail_page = NULL;
|
||||
}
|
||||
}
|
||||
my_queue.head_counter = my_ticket;
|
||||
if( p )
|
||||
base.deallocate_page( p );
|
||||
}
|
||||
};
|
||||
|
||||
struct predicate_leq {
|
||||
ticket t;
|
||||
predicate_leq( ticket t_ ) : t(t_) {}
|
||||
bool operator() ( uintptr_t p ) const {return (ticket)p<=t;}
|
||||
};
|
||||
|
||||
//! Internal representation of a ConcurrentQueue.
|
||||
/** For efficiency, this class has no constructor.
|
||||
The caller is expected to zero-initialize it. */
|
||||
class concurrent_queue_rep {
|
||||
public:
|
||||
private:
|
||||
friend struct micro_queue;
|
||||
|
||||
//! Approximately n_queue/golden ratio
|
||||
static const size_t phi = 3;
|
||||
|
||||
public:
|
||||
//! Must be power of 2
|
||||
static const size_t n_queue = 8;
|
||||
|
||||
//! Map ticket to an array index
|
||||
static size_t index( ticket k ) {
|
||||
return k*phi%n_queue;
|
||||
}
|
||||
|
||||
atomic<ticket> head_counter;
|
||||
concurrent_monitor items_avail;
|
||||
atomic<size_t> n_invalid_entries;
|
||||
char pad1[NFS_MaxLineSize-((sizeof(atomic<ticket>)+sizeof(concurrent_monitor)+sizeof(atomic<size_t>))&(NFS_MaxLineSize-1))];
|
||||
|
||||
atomic<ticket> tail_counter;
|
||||
concurrent_monitor slots_avail;
|
||||
char pad2[NFS_MaxLineSize-((sizeof(atomic<ticket>)+sizeof(concurrent_monitor))&(NFS_MaxLineSize-1))];
|
||||
micro_queue array[n_queue];
|
||||
|
||||
micro_queue& choose( ticket k ) {
|
||||
// The formula here approximates LRU in a cache-oblivious way.
|
||||
return array[index(k)];
|
||||
}
|
||||
|
||||
//! Value for effective_capacity that denotes unbounded queue.
|
||||
static const ptrdiff_t infinite_capacity = ptrdiff_t(~size_t(0)/2);
|
||||
};
|
||||
|
||||
#if _MSC_VER && !defined(__INTEL_COMPILER)
|
||||
// unary minus operator applied to unsigned type, result still unsigned
|
||||
#pragma warning( push )
|
||||
#pragma warning( disable: 4146 )
|
||||
#endif
|
||||
|
||||
static void* invalid_page;
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// micro_queue
|
||||
//------------------------------------------------------------------------
|
||||
void micro_queue::push( const void* item, ticket k, concurrent_queue_base& base,
|
||||
concurrent_queue_base::copy_specifics op_type ) {
|
||||
k &= -concurrent_queue_rep::n_queue;
|
||||
page* p = NULL;
|
||||
// find index on page where we would put the data
|
||||
size_t index = modulo_power_of_two( k/concurrent_queue_rep::n_queue, base.items_per_page );
|
||||
if( !index ) { // make a new page
|
||||
__TBB_TRY {
|
||||
p = base.allocate_page();
|
||||
} __TBB_CATCH(...) {
|
||||
++base.my_rep->n_invalid_entries;
|
||||
make_invalid( k );
|
||||
}
|
||||
p->mask = 0;
|
||||
p->next = NULL;
|
||||
}
|
||||
|
||||
// wait for my turn
|
||||
if( tail_counter!=k ) // The developer insisted on keeping first check out of the backoff loop
|
||||
for( atomic_backoff b(true);;b.pause() ) {
|
||||
ticket tail = tail_counter;
|
||||
if( tail==k ) break;
|
||||
else if( tail&0x1 ) {
|
||||
// no memory. throws an exception; assumes concurrent_queue_rep::n_queue>1
|
||||
++base.my_rep->n_invalid_entries;
|
||||
throw_exception( eid_bad_last_alloc );
|
||||
}
|
||||
}
|
||||
|
||||
if( p ) { // page is newly allocated; insert in micro_queue
|
||||
spin_mutex::scoped_lock lock( page_mutex );
|
||||
if( page* q = tail_page )
|
||||
q->next = p;
|
||||
else
|
||||
head_page = p;
|
||||
tail_page = p;
|
||||
}
|
||||
|
||||
if (item) {
|
||||
p = tail_page;
|
||||
ITT_NOTIFY( sync_acquired, p );
|
||||
__TBB_TRY {
|
||||
if( concurrent_queue_base::copy == op_type ) {
|
||||
base.copy_item( *p, index, item );
|
||||
} else {
|
||||
__TBB_ASSERT( concurrent_queue_base::move == op_type, NULL );
|
||||
static_cast<concurrent_queue_base_v8&>(base).move_item( *p, index, item );
|
||||
}
|
||||
} __TBB_CATCH(...) {
|
||||
++base.my_rep->n_invalid_entries;
|
||||
tail_counter += concurrent_queue_rep::n_queue;
|
||||
__TBB_RETHROW();
|
||||
}
|
||||
ITT_NOTIFY( sync_releasing, p );
|
||||
// If no exception was thrown, mark item as present.
|
||||
p->mask |= uintptr_t(1)<<index;
|
||||
}
|
||||
else // no item; this was called from abort_push
|
||||
++base.my_rep->n_invalid_entries;
|
||||
|
||||
tail_counter += concurrent_queue_rep::n_queue;
|
||||
}
|
||||
|
||||
|
||||
void micro_queue::abort_push( ticket k, concurrent_queue_base& base ) {
|
||||
push(NULL, k, base, concurrent_queue_base::copy);
|
||||
}
|
||||
|
||||
bool micro_queue::pop( void* dst, ticket k, concurrent_queue_base& base ) {
|
||||
k &= -concurrent_queue_rep::n_queue;
|
||||
spin_wait_until_eq( head_counter, k );
|
||||
spin_wait_while_eq( tail_counter, k );
|
||||
page& p = *head_page;
|
||||
__TBB_ASSERT( &p, NULL );
|
||||
size_t index = modulo_power_of_two( k/concurrent_queue_rep::n_queue, base.items_per_page );
|
||||
bool success = false;
|
||||
{
|
||||
micro_queue_pop_finalizer finalizer( *this, base, k+concurrent_queue_rep::n_queue, index==base.items_per_page-1 ? &p : NULL );
|
||||
if( p.mask & uintptr_t(1)<<index ) {
|
||||
success = true;
|
||||
ITT_NOTIFY( sync_acquired, dst );
|
||||
ITT_NOTIFY( sync_acquired, head_page );
|
||||
base.assign_and_destroy_item( dst, p, index );
|
||||
ITT_NOTIFY( sync_releasing, head_page );
|
||||
} else {
|
||||
--base.my_rep->n_invalid_entries;
|
||||
}
|
||||
}
|
||||
return success;
|
||||
}
|
||||
|
||||
micro_queue& micro_queue::assign( const micro_queue& src, concurrent_queue_base& base,
|
||||
concurrent_queue_base::copy_specifics op_type )
|
||||
{
|
||||
head_counter = src.head_counter;
|
||||
tail_counter = src.tail_counter;
|
||||
|
||||
const page* srcp = src.head_page;
|
||||
if( srcp ) {
|
||||
ticket g_index = head_counter;
|
||||
__TBB_TRY {
|
||||
size_t n_items = (tail_counter-head_counter)/concurrent_queue_rep::n_queue;
|
||||
size_t index = modulo_power_of_two( head_counter/concurrent_queue_rep::n_queue, base.items_per_page );
|
||||
size_t end_in_first_page = (index+n_items<base.items_per_page)?(index+n_items):base.items_per_page;
|
||||
|
||||
head_page = make_copy( base, srcp, index, end_in_first_page, g_index, op_type );
|
||||
page* cur_page = head_page;
|
||||
|
||||
if( srcp != src.tail_page ) {
|
||||
for( srcp = srcp->next; srcp!=src.tail_page; srcp=srcp->next ) {
|
||||
cur_page->next = make_copy( base, srcp, 0, base.items_per_page, g_index, op_type );
|
||||
cur_page = cur_page->next;
|
||||
}
|
||||
|
||||
__TBB_ASSERT( srcp==src.tail_page, NULL );
|
||||
|
||||
size_t last_index = modulo_power_of_two( tail_counter/concurrent_queue_rep::n_queue, base.items_per_page );
|
||||
if( last_index==0 ) last_index = base.items_per_page;
|
||||
|
||||
cur_page->next = make_copy( base, srcp, 0, last_index, g_index, op_type );
|
||||
cur_page = cur_page->next;
|
||||
}
|
||||
tail_page = cur_page;
|
||||
} __TBB_CATCH(...) {
|
||||
make_invalid( g_index );
|
||||
}
|
||||
} else {
|
||||
head_page = tail_page = NULL;
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
|
||||
concurrent_queue_base::page* micro_queue::make_copy( concurrent_queue_base& base,
|
||||
const concurrent_queue_base::page* src_page, size_t begin_in_page, size_t end_in_page,
|
||||
ticket& g_index, concurrent_queue_base::copy_specifics op_type )
|
||||
{
|
||||
page* new_page = base.allocate_page();
|
||||
new_page->next = NULL;
|
||||
new_page->mask = src_page->mask;
|
||||
for( ; begin_in_page!=end_in_page; ++begin_in_page, ++g_index )
|
||||
if( new_page->mask & uintptr_t(1)<<begin_in_page )
|
||||
if( concurrent_queue_base::copy == op_type ) {
|
||||
base.copy_page_item( *new_page, begin_in_page, *src_page, begin_in_page );
|
||||
} else {
|
||||
__TBB_ASSERT( concurrent_queue_base::move == op_type, NULL );
|
||||
static_cast<concurrent_queue_base_v8&>(base).move_page_item( *new_page, begin_in_page, *src_page, begin_in_page );
|
||||
}
|
||||
return new_page;
|
||||
}
|
||||
|
||||
void micro_queue::make_invalid( ticket k )
|
||||
{
|
||||
static concurrent_queue_base::page dummy = {static_cast<page*>((void*)1), 0};
|
||||
// mark it so that no more pushes are allowed.
|
||||
invalid_page = &dummy;
|
||||
{
|
||||
spin_mutex::scoped_lock lock( page_mutex );
|
||||
tail_counter = k+concurrent_queue_rep::n_queue+1;
|
||||
if( page* q = tail_page )
|
||||
q->next = static_cast<page*>(invalid_page);
|
||||
else
|
||||
head_page = static_cast<page*>(invalid_page);
|
||||
tail_page = static_cast<page*>(invalid_page);
|
||||
}
|
||||
__TBB_RETHROW();
|
||||
}
|
||||
|
||||
#if _MSC_VER && !defined(__INTEL_COMPILER)
|
||||
#pragma warning( pop )
|
||||
#endif // warning 4146 is back
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// concurrent_queue_base
|
||||
//------------------------------------------------------------------------
|
||||
concurrent_queue_base_v3::concurrent_queue_base_v3( size_t item_sz ) {
|
||||
items_per_page = item_sz<= 8 ? 32 :
|
||||
item_sz<= 16 ? 16 :
|
||||
item_sz<= 32 ? 8 :
|
||||
item_sz<= 64 ? 4 :
|
||||
item_sz<=128 ? 2 :
|
||||
1;
|
||||
my_capacity = size_t(-1)/(item_sz>1 ? item_sz : 2);
|
||||
my_rep = cache_aligned_allocator<concurrent_queue_rep>().allocate(1);
|
||||
__TBB_ASSERT( (size_t)my_rep % NFS_GetLineSize()==0, "alignment error" );
|
||||
__TBB_ASSERT( (size_t)&my_rep->head_counter % NFS_GetLineSize()==0, "alignment error" );
|
||||
__TBB_ASSERT( (size_t)&my_rep->tail_counter % NFS_GetLineSize()==0, "alignment error" );
|
||||
__TBB_ASSERT( (size_t)&my_rep->array % NFS_GetLineSize()==0, "alignment error" );
|
||||
memset(my_rep,0,sizeof(concurrent_queue_rep));
|
||||
new ( &my_rep->items_avail ) concurrent_monitor();
|
||||
new ( &my_rep->slots_avail ) concurrent_monitor();
|
||||
this->item_size = item_sz;
|
||||
}
|
||||
|
||||
concurrent_queue_base_v3::~concurrent_queue_base_v3() {
|
||||
size_t nq = my_rep->n_queue;
|
||||
for( size_t i=0; i<nq; i++ )
|
||||
__TBB_ASSERT( my_rep->array[i].tail_page==NULL, "pages were not freed properly" );
|
||||
cache_aligned_allocator<concurrent_queue_rep>().deallocate(my_rep,1);
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_push( const void* src ) {
|
||||
internal_insert_item( src, copy );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v8::internal_push_move( const void* src ) {
|
||||
internal_insert_item( src, move );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_insert_item( const void* src, copy_specifics op_type ) {
|
||||
concurrent_queue_rep& r = *my_rep;
|
||||
ticket k = r.tail_counter++;
|
||||
ptrdiff_t e = my_capacity;
|
||||
#if DO_ITT_NOTIFY
|
||||
bool sync_prepare_done = false;
|
||||
#endif
|
||||
if( (ptrdiff_t)(k-r.head_counter)>=e ) { // queue is full
|
||||
#if DO_ITT_NOTIFY
|
||||
if( !sync_prepare_done ) {
|
||||
ITT_NOTIFY( sync_prepare, &sync_prepare_done );
|
||||
sync_prepare_done = true;
|
||||
}
|
||||
#endif
|
||||
bool slept = false;
|
||||
concurrent_monitor::thread_context thr_ctx;
|
||||
r.slots_avail.prepare_wait( thr_ctx, ((ptrdiff_t)(k-e)) );
|
||||
while( (ptrdiff_t)(k-r.head_counter)>=const_cast<volatile ptrdiff_t&>(e = my_capacity) ) {
|
||||
__TBB_TRY {
|
||||
slept = r.slots_avail.commit_wait( thr_ctx );
|
||||
} __TBB_CATCH( tbb::user_abort& ) {
|
||||
r.choose(k).abort_push(k, *this);
|
||||
__TBB_RETHROW();
|
||||
} __TBB_CATCH(...) {
|
||||
__TBB_RETHROW();
|
||||
}
|
||||
if (slept == true) break;
|
||||
r.slots_avail.prepare_wait( thr_ctx, ((ptrdiff_t)(k-e)) );
|
||||
}
|
||||
if( !slept )
|
||||
r.slots_avail.cancel_wait( thr_ctx );
|
||||
}
|
||||
ITT_NOTIFY( sync_acquired, &sync_prepare_done );
|
||||
__TBB_ASSERT( (ptrdiff_t)(k-r.head_counter)<my_capacity, NULL);
|
||||
r.choose( k ).push( src, k, *this, op_type );
|
||||
r.items_avail.notify( predicate_leq(k) );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_pop( void* dst ) {
|
||||
concurrent_queue_rep& r = *my_rep;
|
||||
ticket k;
|
||||
#if DO_ITT_NOTIFY
|
||||
bool sync_prepare_done = false;
|
||||
#endif
|
||||
do {
|
||||
k=r.head_counter++;
|
||||
if ( (ptrdiff_t)(r.tail_counter-k)<=0 ) { // queue is empty
|
||||
#if DO_ITT_NOTIFY
|
||||
if( !sync_prepare_done ) {
|
||||
ITT_NOTIFY( sync_prepare, dst );
|
||||
sync_prepare_done = true;
|
||||
}
|
||||
#endif
|
||||
bool slept = false;
|
||||
concurrent_monitor::thread_context thr_ctx;
|
||||
r.items_avail.prepare_wait( thr_ctx, k );
|
||||
while( (ptrdiff_t)(r.tail_counter-k)<=0 ) {
|
||||
__TBB_TRY {
|
||||
slept = r.items_avail.commit_wait( thr_ctx );
|
||||
} __TBB_CATCH( tbb::user_abort& ) {
|
||||
r.head_counter--;
|
||||
__TBB_RETHROW();
|
||||
} __TBB_CATCH(...) {
|
||||
__TBB_RETHROW();
|
||||
}
|
||||
if (slept == true) break;
|
||||
r.items_avail.prepare_wait( thr_ctx, k );
|
||||
}
|
||||
if( !slept )
|
||||
r.items_avail.cancel_wait( thr_ctx );
|
||||
}
|
||||
__TBB_ASSERT((ptrdiff_t)(r.tail_counter-k)>0, NULL);
|
||||
} while( !r.choose(k).pop(dst,k,*this) );
|
||||
|
||||
// wake up a producer..
|
||||
r.slots_avail.notify( predicate_leq(k) );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_abort() {
|
||||
concurrent_queue_rep& r = *my_rep;
|
||||
r.items_avail.abort_all();
|
||||
r.slots_avail.abort_all();
|
||||
}
|
||||
|
||||
bool concurrent_queue_base_v3::internal_pop_if_present( void* dst ) {
|
||||
concurrent_queue_rep& r = *my_rep;
|
||||
ticket k;
|
||||
do {
|
||||
k = r.head_counter;
|
||||
for(;;) {
|
||||
if( (ptrdiff_t)(r.tail_counter-k)<=0 ) {
|
||||
// Queue is empty
|
||||
return false;
|
||||
}
|
||||
// Queue had item with ticket k when we looked. Attempt to get that item.
|
||||
ticket tk=k;
|
||||
k = r.head_counter.compare_and_swap( tk+1, tk );
|
||||
if( k==tk )
|
||||
break;
|
||||
// Another thread snatched the item, retry.
|
||||
}
|
||||
} while( !r.choose( k ).pop( dst, k, *this ) );
|
||||
|
||||
r.slots_avail.notify( predicate_leq(k) );
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool concurrent_queue_base_v3::internal_push_if_not_full( const void* src ) {
|
||||
return internal_insert_if_not_full( src, copy );
|
||||
}
|
||||
|
||||
bool concurrent_queue_base_v8::internal_push_move_if_not_full( const void* src ) {
|
||||
return internal_insert_if_not_full( src, move );
|
||||
}
|
||||
|
||||
bool concurrent_queue_base_v3::internal_insert_if_not_full( const void* src, copy_specifics op_type ) {
|
||||
concurrent_queue_rep& r = *my_rep;
|
||||
ticket k = r.tail_counter;
|
||||
for(;;) {
|
||||
if( (ptrdiff_t)(k-r.head_counter)>=my_capacity ) {
|
||||
// Queue is full
|
||||
return false;
|
||||
}
|
||||
// Queue had empty slot with ticket k when we looked. Attempt to claim that slot.
|
||||
ticket tk=k;
|
||||
k = r.tail_counter.compare_and_swap( tk+1, tk );
|
||||
if( k==tk )
|
||||
break;
|
||||
// Another thread claimed the slot, so retry.
|
||||
}
|
||||
r.choose(k).push(src, k, *this, op_type);
|
||||
r.items_avail.notify( predicate_leq(k) );
|
||||
return true;
|
||||
}
|
||||
|
||||
ptrdiff_t concurrent_queue_base_v3::internal_size() const {
|
||||
__TBB_ASSERT( sizeof(ptrdiff_t)<=sizeof(size_t), NULL );
|
||||
return ptrdiff_t(my_rep->tail_counter-my_rep->head_counter-my_rep->n_invalid_entries);
|
||||
}
|
||||
|
||||
bool concurrent_queue_base_v3::internal_empty() const {
|
||||
ticket tc = my_rep->tail_counter;
|
||||
ticket hc = my_rep->head_counter;
|
||||
// if tc!=r.tail_counter, the queue was not empty at some point between the two reads.
|
||||
return ( tc==my_rep->tail_counter && ptrdiff_t(tc-hc-my_rep->n_invalid_entries)<=0 );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_set_capacity( ptrdiff_t capacity, size_t /*item_sz*/ ) {
|
||||
my_capacity = capacity<0 ? concurrent_queue_rep::infinite_capacity : capacity;
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_finish_clear() {
|
||||
size_t nq = my_rep->n_queue;
|
||||
for( size_t i=0; i<nq; ++i ) {
|
||||
page* tp = my_rep->array[i].tail_page;
|
||||
__TBB_ASSERT( my_rep->array[i].head_page==tp, "at most one page should remain" );
|
||||
if( tp!=NULL) {
|
||||
if( tp!=invalid_page ) deallocate_page( tp );
|
||||
my_rep->array[i].tail_page = NULL;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_throw_exception() const {
|
||||
throw_exception( eid_bad_alloc );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::internal_assign( const concurrent_queue_base& src, copy_specifics op_type ) {
|
||||
items_per_page = src.items_per_page;
|
||||
my_capacity = src.my_capacity;
|
||||
|
||||
// copy concurrent_queue_rep.
|
||||
my_rep->head_counter = src.my_rep->head_counter;
|
||||
my_rep->tail_counter = src.my_rep->tail_counter;
|
||||
my_rep->n_invalid_entries = src.my_rep->n_invalid_entries;
|
||||
|
||||
// copy micro_queues
|
||||
for( size_t i = 0; i<my_rep->n_queue; ++i )
|
||||
my_rep->array[i].assign( src.my_rep->array[i], *this, op_type );
|
||||
|
||||
__TBB_ASSERT( my_rep->head_counter==src.my_rep->head_counter && my_rep->tail_counter==src.my_rep->tail_counter,
|
||||
"the source concurrent queue should not be concurrently modified." );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v3::assign( const concurrent_queue_base& src ) {
|
||||
internal_assign( src, copy );
|
||||
}
|
||||
|
||||
void concurrent_queue_base_v8::move_content( concurrent_queue_base_v8& src ) {
|
||||
internal_assign( src, move );
|
||||
}
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// concurrent_queue_iterator_rep
|
||||
//------------------------------------------------------------------------
|
||||
class concurrent_queue_iterator_rep: no_assign {
|
||||
public:
|
||||
ticket head_counter;
|
||||
const concurrent_queue_base& my_queue;
|
||||
const size_t offset_of_last;
|
||||
concurrent_queue_base::page* array[concurrent_queue_rep::n_queue];
|
||||
concurrent_queue_iterator_rep( const concurrent_queue_base& queue, size_t offset_of_last_ ) :
|
||||
head_counter(queue.my_rep->head_counter),
|
||||
my_queue(queue),
|
||||
offset_of_last(offset_of_last_)
|
||||
{
|
||||
const concurrent_queue_rep& rep = *queue.my_rep;
|
||||
for( size_t k=0; k<concurrent_queue_rep::n_queue; ++k )
|
||||
array[k] = rep.array[k].head_page;
|
||||
}
|
||||
//! Set item to point to kth element. Return true if at end of queue or item is marked valid; false otherwise.
|
||||
bool get_item( void*& item, size_t k ) {
|
||||
if( k==my_queue.my_rep->tail_counter ) {
|
||||
item = NULL;
|
||||
return true;
|
||||
} else {
|
||||
concurrent_queue_base::page* p = array[concurrent_queue_rep::index(k)];
|
||||
__TBB_ASSERT(p,NULL);
|
||||
size_t i = modulo_power_of_two( k/concurrent_queue_rep::n_queue, my_queue.items_per_page );
|
||||
item = static_cast<unsigned char*>(static_cast<void*>(p)) + offset_of_last + my_queue.item_size*i;
|
||||
return (p->mask & uintptr_t(1)<<i)!=0;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// concurrent_queue_iterator_base
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
void concurrent_queue_iterator_base_v3::initialize( const concurrent_queue_base& queue, size_t offset_of_last ) {
|
||||
my_rep = cache_aligned_allocator<concurrent_queue_iterator_rep>().allocate(1);
|
||||
new( my_rep ) concurrent_queue_iterator_rep(queue,offset_of_last);
|
||||
size_t k = my_rep->head_counter;
|
||||
if( !my_rep->get_item(my_item, k) ) advance();
|
||||
}
|
||||
|
||||
concurrent_queue_iterator_base_v3::concurrent_queue_iterator_base_v3( const concurrent_queue_base& queue ) {
|
||||
initialize(queue,0);
|
||||
}
|
||||
|
||||
concurrent_queue_iterator_base_v3::concurrent_queue_iterator_base_v3( const concurrent_queue_base& queue, size_t offset_of_last ) {
|
||||
initialize(queue,offset_of_last);
|
||||
}
|
||||
|
||||
void concurrent_queue_iterator_base_v3::assign( const concurrent_queue_iterator_base& other ) {
|
||||
if( my_rep!=other.my_rep ) {
|
||||
if( my_rep ) {
|
||||
cache_aligned_allocator<concurrent_queue_iterator_rep>().deallocate(my_rep, 1);
|
||||
my_rep = NULL;
|
||||
}
|
||||
if( other.my_rep ) {
|
||||
my_rep = cache_aligned_allocator<concurrent_queue_iterator_rep>().allocate(1);
|
||||
new( my_rep ) concurrent_queue_iterator_rep( *other.my_rep );
|
||||
}
|
||||
}
|
||||
my_item = other.my_item;
|
||||
}
|
||||
|
||||
void concurrent_queue_iterator_base_v3::advance() {
|
||||
__TBB_ASSERT( my_item, "attempt to increment iterator past end of queue" );
|
||||
size_t k = my_rep->head_counter;
|
||||
const concurrent_queue_base& queue = my_rep->my_queue;
|
||||
#if TBB_USE_ASSERT
|
||||
void* tmp;
|
||||
my_rep->get_item(tmp,k);
|
||||
__TBB_ASSERT( my_item==tmp, NULL );
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
size_t i = modulo_power_of_two( k/concurrent_queue_rep::n_queue, queue.items_per_page );
|
||||
if( i==queue.items_per_page-1 ) {
|
||||
concurrent_queue_base::page*& root = my_rep->array[concurrent_queue_rep::index(k)];
|
||||
root = root->next;
|
||||
}
|
||||
// advance k
|
||||
my_rep->head_counter = ++k;
|
||||
if( !my_rep->get_item(my_item, k) ) advance();
|
||||
}
|
||||
|
||||
concurrent_queue_iterator_base_v3::~concurrent_queue_iterator_base_v3() {
|
||||
//delete my_rep;
|
||||
cache_aligned_allocator<concurrent_queue_iterator_rep>().deallocate(my_rep, 1);
|
||||
my_rep = NULL;
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
|
||||
} // namespace tbb
|
||||
@@ -0,0 +1,462 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_concurrent_queue_H
|
||||
#define __TBB_concurrent_queue_H
|
||||
|
||||
#include "internal/_concurrent_queue_impl.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace strict_ppl {
|
||||
|
||||
//! A high-performance thread-safe non-blocking concurrent queue.
|
||||
/** Multiple threads may each push and pop concurrently.
|
||||
Assignment construction is not allowed.
|
||||
@ingroup containers */
|
||||
template<typename T, typename A = cache_aligned_allocator<T> >
|
||||
class concurrent_queue: public internal::concurrent_queue_base_v3<T> {
|
||||
template<typename Container, typename Value> friend class internal::concurrent_queue_iterator;
|
||||
|
||||
//! Allocator type
|
||||
typedef typename A::template rebind<char>::other page_allocator_type;
|
||||
page_allocator_type my_allocator;
|
||||
|
||||
//! Allocates a block of size n (bytes)
|
||||
/*override*/ virtual void *allocate_block( size_t n ) {
|
||||
void *b = reinterpret_cast<void*>(my_allocator.allocate( n ));
|
||||
if( !b )
|
||||
internal::throw_exception(internal::eid_bad_alloc);
|
||||
return b;
|
||||
}
|
||||
|
||||
//! Deallocates block created by allocate_block.
|
||||
/*override*/ virtual void deallocate_block( void *b, size_t n ) {
|
||||
my_allocator.deallocate( reinterpret_cast<char*>(b), n );
|
||||
}
|
||||
|
||||
static void copy_construct_item(T* location, const void* src){
|
||||
new (location) T(*static_cast<const T*>(src));
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
static void move_construct_item(T* location, const void* src) {
|
||||
new (location) T( std::move(*static_cast<T*>(const_cast<void*>(src))) );
|
||||
}
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
public:
|
||||
//! Element type in the queue.
|
||||
typedef T value_type;
|
||||
|
||||
//! Reference type
|
||||
typedef T& reference;
|
||||
|
||||
//! Const reference type
|
||||
typedef const T& const_reference;
|
||||
|
||||
//! Integral type for representing size of the queue.
|
||||
typedef size_t size_type;
|
||||
|
||||
//! Difference type for iterator
|
||||
typedef ptrdiff_t difference_type;
|
||||
|
||||
//! Allocator type
|
||||
typedef A allocator_type;
|
||||
|
||||
//! Construct empty queue
|
||||
explicit concurrent_queue(const allocator_type& a = allocator_type()) :
|
||||
my_allocator( a )
|
||||
{
|
||||
}
|
||||
|
||||
//! [begin,end) constructor
|
||||
template<typename InputIterator>
|
||||
concurrent_queue( InputIterator begin, InputIterator end, const allocator_type& a = allocator_type()) :
|
||||
my_allocator( a )
|
||||
{
|
||||
for( ; begin != end; ++begin )
|
||||
this->push(*begin);
|
||||
}
|
||||
|
||||
//! Copy constructor
|
||||
concurrent_queue( const concurrent_queue& src, const allocator_type& a = allocator_type()) :
|
||||
internal::concurrent_queue_base_v3<T>(), my_allocator( a )
|
||||
{
|
||||
this->assign( src, copy_construct_item );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Move constructors
|
||||
concurrent_queue( concurrent_queue&& src ) :
|
||||
internal::concurrent_queue_base_v3<T>(), my_allocator( std::move(src.my_allocator) )
|
||||
{
|
||||
this->internal_swap( src );
|
||||
}
|
||||
|
||||
concurrent_queue( concurrent_queue&& src, const allocator_type& a ) :
|
||||
internal::concurrent_queue_base_v3<T>(), my_allocator( a )
|
||||
{
|
||||
// checking that memory allocated by one instance of allocator can be deallocated
|
||||
// with another
|
||||
if( my_allocator == src.my_allocator) {
|
||||
this->internal_swap( src );
|
||||
} else {
|
||||
// allocators are different => performing per-element move
|
||||
this->assign( src, move_construct_item );
|
||||
src.clear();
|
||||
}
|
||||
}
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! Destroy queue
|
||||
~concurrent_queue();
|
||||
|
||||
//! Enqueue an item at tail of queue.
|
||||
void push( const T& source ) {
|
||||
this->internal_push( &source, copy_construct_item );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
void push( T&& source ) {
|
||||
this->internal_push( &source, move_construct_item );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
|
||||
template<typename... Arguments>
|
||||
void emplace( Arguments&&... args ) {
|
||||
push( T(std::forward<Arguments>( args )...) );
|
||||
}
|
||||
#endif //__TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! Attempt to dequeue an item from head of queue.
|
||||
/** Does not wait for item to become available.
|
||||
Returns true if successful; false otherwise. */
|
||||
bool try_pop( T& result ) {
|
||||
return this->internal_try_pop( &result );
|
||||
}
|
||||
|
||||
//! Return the number of items in the queue; thread unsafe
|
||||
size_type unsafe_size() const {return this->internal_size();}
|
||||
|
||||
//! Equivalent to size()==0.
|
||||
bool empty() const {return this->internal_empty();}
|
||||
|
||||
//! Clear the queue. not thread-safe.
|
||||
void clear() ;
|
||||
|
||||
//! Return allocator object
|
||||
allocator_type get_allocator() const { return this->my_allocator; }
|
||||
|
||||
typedef internal::concurrent_queue_iterator<concurrent_queue,T> iterator;
|
||||
typedef internal::concurrent_queue_iterator<concurrent_queue,const T> const_iterator;
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// The iterators are intended only for debugging. They are slow and not thread safe.
|
||||
//------------------------------------------------------------------------
|
||||
iterator unsafe_begin() {return iterator(*this);}
|
||||
iterator unsafe_end() {return iterator();}
|
||||
const_iterator unsafe_begin() const {return const_iterator(*this);}
|
||||
const_iterator unsafe_end() const {return const_iterator();}
|
||||
} ;
|
||||
|
||||
template<typename T, class A>
|
||||
concurrent_queue<T,A>::~concurrent_queue() {
|
||||
clear();
|
||||
this->internal_finish_clear();
|
||||
}
|
||||
|
||||
template<typename T, class A>
|
||||
void concurrent_queue<T,A>::clear() {
|
||||
while( !empty() ) {
|
||||
T value;
|
||||
this->internal_try_pop(&value);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace strict_ppl
|
||||
|
||||
//! A high-performance thread-safe blocking concurrent bounded queue.
|
||||
/** This is the pre-PPL TBB concurrent queue which supports boundedness and blocking semantics.
|
||||
Note that method names agree with the PPL-style concurrent queue.
|
||||
Multiple threads may each push and pop concurrently.
|
||||
Assignment construction is not allowed.
|
||||
@ingroup containers */
|
||||
template<typename T, class A = cache_aligned_allocator<T> >
|
||||
class concurrent_bounded_queue: public internal::concurrent_queue_base_v8 {
|
||||
template<typename Container, typename Value> friend class internal::concurrent_queue_iterator;
|
||||
|
||||
//! Allocator type
|
||||
typedef typename A::template rebind<char>::other page_allocator_type;
|
||||
page_allocator_type my_allocator;
|
||||
|
||||
typedef typename concurrent_queue_base_v3::padded_page<T> padded_page;
|
||||
typedef typename concurrent_queue_base_v3::copy_specifics copy_specifics;
|
||||
|
||||
//! Class used to ensure exception-safety of method "pop"
|
||||
class destroyer: internal::no_copy {
|
||||
T& my_value;
|
||||
public:
|
||||
destroyer( T& value ) : my_value(value) {}
|
||||
~destroyer() {my_value.~T();}
|
||||
};
|
||||
|
||||
T& get_ref( page& p, size_t index ) {
|
||||
__TBB_ASSERT( index<items_per_page, NULL );
|
||||
return (&static_cast<padded_page*>(static_cast<void*>(&p))->last)[index];
|
||||
}
|
||||
|
||||
/*override*/ virtual void copy_item( page& dst, size_t index, const void* src ) {
|
||||
new( &get_ref(dst,index) ) T(*static_cast<const T*>(src));
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
/*override*/ virtual void move_item( page& dst, size_t index, const void* src ) {
|
||||
new( &get_ref(dst,index) ) T( std::move(*static_cast<T*>(const_cast<void*>(src))) );
|
||||
}
|
||||
#else
|
||||
/*override*/ virtual void move_item( page&, size_t, const void* ) {
|
||||
__TBB_ASSERT( false, "Unreachable code" );
|
||||
}
|
||||
#endif
|
||||
|
||||
/*override*/ virtual void copy_page_item( page& dst, size_t dindex, const page& src, size_t sindex ) {
|
||||
new( &get_ref(dst,dindex) ) T( get_ref( const_cast<page&>(src), sindex ) );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
/*override*/ virtual void move_page_item( page& dst, size_t dindex, const page& src, size_t sindex ) {
|
||||
new( &get_ref(dst,dindex) ) T( std::move(get_ref( const_cast<page&>(src), sindex )) );
|
||||
}
|
||||
#else
|
||||
/*override*/ virtual void move_page_item( page&, size_t, const page&, size_t ) {
|
||||
__TBB_ASSERT( false, "Unreachable code" );
|
||||
}
|
||||
#endif
|
||||
|
||||
/*override*/ virtual void assign_and_destroy_item( void* dst, page& src, size_t index ) {
|
||||
T& from = get_ref(src,index);
|
||||
destroyer d(from);
|
||||
*static_cast<T*>(dst) = tbb::internal::move( from );
|
||||
}
|
||||
|
||||
/*override*/ virtual page *allocate_page() {
|
||||
size_t n = sizeof(padded_page) + (items_per_page-1)*sizeof(T);
|
||||
page *p = reinterpret_cast<page*>(my_allocator.allocate( n ));
|
||||
if( !p )
|
||||
internal::throw_exception(internal::eid_bad_alloc);
|
||||
return p;
|
||||
}
|
||||
|
||||
/*override*/ virtual void deallocate_page( page *p ) {
|
||||
size_t n = sizeof(padded_page) + (items_per_page-1)*sizeof(T);
|
||||
my_allocator.deallocate( reinterpret_cast<char*>(p), n );
|
||||
}
|
||||
|
||||
public:
|
||||
//! Element type in the queue.
|
||||
typedef T value_type;
|
||||
|
||||
//! Allocator type
|
||||
typedef A allocator_type;
|
||||
|
||||
//! Reference type
|
||||
typedef T& reference;
|
||||
|
||||
//! Const reference type
|
||||
typedef const T& const_reference;
|
||||
|
||||
//! Integral type for representing size of the queue.
|
||||
/** Note that the size_type is a signed integral type.
|
||||
This is because the size can be negative if there are pending pops without corresponding pushes. */
|
||||
typedef std::ptrdiff_t size_type;
|
||||
|
||||
//! Difference type for iterator
|
||||
typedef std::ptrdiff_t difference_type;
|
||||
|
||||
//! Construct empty queue
|
||||
explicit concurrent_bounded_queue(const allocator_type& a = allocator_type()) :
|
||||
concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
|
||||
{
|
||||
}
|
||||
|
||||
//! Copy constructor
|
||||
concurrent_bounded_queue( const concurrent_bounded_queue& src, const allocator_type& a = allocator_type())
|
||||
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
|
||||
{
|
||||
assign( src );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Move constructors
|
||||
concurrent_bounded_queue( concurrent_bounded_queue&& src )
|
||||
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( std::move(src.my_allocator) )
|
||||
{
|
||||
internal_swap( src );
|
||||
}
|
||||
|
||||
concurrent_bounded_queue( concurrent_bounded_queue&& src, const allocator_type& a )
|
||||
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
|
||||
{
|
||||
// checking that memory allocated by one instance of allocator can be deallocated
|
||||
// with another
|
||||
if( my_allocator == src.my_allocator) {
|
||||
this->internal_swap( src );
|
||||
} else {
|
||||
// allocators are different => performing per-element move
|
||||
this->move_content( src );
|
||||
src.clear();
|
||||
}
|
||||
}
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! [begin,end) constructor
|
||||
template<typename InputIterator>
|
||||
concurrent_bounded_queue( InputIterator begin, InputIterator end,
|
||||
const allocator_type& a = allocator_type())
|
||||
: concurrent_queue_base_v8( sizeof(T) ), my_allocator( a )
|
||||
{
|
||||
for( ; begin != end; ++begin )
|
||||
internal_push_if_not_full(&*begin);
|
||||
}
|
||||
|
||||
//! Destroy queue
|
||||
~concurrent_bounded_queue();
|
||||
|
||||
//! Enqueue an item at tail of queue.
|
||||
void push( const T& source ) {
|
||||
internal_push( &source );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Move an item at tail of queue.
|
||||
void push( T&& source ) {
|
||||
internal_push_move( &source );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
|
||||
template<typename... Arguments>
|
||||
void emplace( Arguments&&... args ) {
|
||||
push( T(std::forward<Arguments>( args )...) );
|
||||
}
|
||||
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! Dequeue item from head of queue.
|
||||
/** Block until an item becomes available, and then dequeue it. */
|
||||
void pop( T& destination ) {
|
||||
internal_pop( &destination );
|
||||
}
|
||||
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
//! Abort all pending queue operations
|
||||
void abort() {
|
||||
internal_abort();
|
||||
}
|
||||
#endif
|
||||
|
||||
//! Enqueue an item at tail of queue if queue is not already full.
|
||||
/** Does not wait for queue to become not full.
|
||||
Returns true if item is pushed; false if queue was already full. */
|
||||
bool try_push( const T& source ) {
|
||||
return internal_push_if_not_full( &source );
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
//! Move an item at tail of queue if queue is not already full.
|
||||
/** Does not wait for queue to become not full.
|
||||
Returns true if item is pushed; false if queue was already full. */
|
||||
bool try_push( T&& source ) {
|
||||
return internal_push_move_if_not_full( &source );
|
||||
}
|
||||
#if __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT
|
||||
template<typename... Arguments>
|
||||
bool try_emplace( Arguments&&... args ) {
|
||||
return try_push( T(std::forward<Arguments>( args )...) );
|
||||
}
|
||||
#endif /* __TBB_CPP11_VARIADIC_TEMPLATES_PRESENT */
|
||||
#endif /* __TBB_CPP11_RVALUE_REF_PRESENT */
|
||||
|
||||
//! Attempt to dequeue an item from head of queue.
|
||||
/** Does not wait for item to become available.
|
||||
Returns true if successful; false otherwise. */
|
||||
bool try_pop( T& destination ) {
|
||||
return internal_pop_if_present( &destination );
|
||||
}
|
||||
|
||||
//! Return number of pushes minus number of pops.
|
||||
/** Note that the result can be negative if there are pops waiting for the
|
||||
corresponding pushes. The result can also exceed capacity() if there
|
||||
are push operations in flight. */
|
||||
size_type size() const {return internal_size();}
|
||||
|
||||
//! Equivalent to size()<=0.
|
||||
bool empty() const {return internal_empty();}
|
||||
|
||||
//! Maximum number of allowed elements
|
||||
size_type capacity() const {
|
||||
return my_capacity;
|
||||
}
|
||||
|
||||
//! Set the capacity
|
||||
/** Setting the capacity to 0 causes subsequent try_push operations to always fail,
|
||||
and subsequent push operations to block forever. */
|
||||
void set_capacity( size_type new_capacity ) {
|
||||
internal_set_capacity( new_capacity, sizeof(T) );
|
||||
}
|
||||
|
||||
//! return allocator object
|
||||
allocator_type get_allocator() const { return this->my_allocator; }
|
||||
|
||||
//! clear the queue. not thread-safe.
|
||||
void clear() ;
|
||||
|
||||
typedef internal::concurrent_queue_iterator<concurrent_bounded_queue,T> iterator;
|
||||
typedef internal::concurrent_queue_iterator<concurrent_bounded_queue,const T> const_iterator;
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// The iterators are intended only for debugging. They are slow and not thread safe.
|
||||
//------------------------------------------------------------------------
|
||||
iterator unsafe_begin() {return iterator(*this);}
|
||||
iterator unsafe_end() {return iterator();}
|
||||
const_iterator unsafe_begin() const {return const_iterator(*this);}
|
||||
const_iterator unsafe_end() const {return const_iterator();}
|
||||
|
||||
};
|
||||
|
||||
template<typename T, class A>
|
||||
concurrent_bounded_queue<T,A>::~concurrent_bounded_queue() {
|
||||
clear();
|
||||
internal_finish_clear();
|
||||
}
|
||||
|
||||
template<typename T, class A>
|
||||
void concurrent_bounded_queue<T,A>::clear() {
|
||||
while( !empty() ) {
|
||||
T value;
|
||||
internal_pop_if_present(&value);
|
||||
}
|
||||
}
|
||||
|
||||
using strict_ppl::concurrent_queue;
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_concurrent_queue_H */
|
||||
@@ -0,0 +1,326 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
/* Container implementations in this header are based on PPL implementations
|
||||
provided by Microsoft. */
|
||||
|
||||
#ifndef __TBB_concurrent_unordered_map_H
|
||||
#define __TBB_concurrent_unordered_map_H
|
||||
|
||||
#include "internal/_concurrent_unordered_impl.h"
|
||||
|
||||
namespace tbb
|
||||
{
|
||||
|
||||
namespace interface5 {
|
||||
|
||||
// Template class for hash map traits
|
||||
template<typename Key, typename T, typename Hash_compare, typename Allocator, bool Allow_multimapping>
|
||||
class concurrent_unordered_map_traits
|
||||
{
|
||||
protected:
|
||||
typedef std::pair<const Key, T> value_type;
|
||||
typedef Key key_type;
|
||||
typedef Hash_compare hash_compare;
|
||||
typedef typename Allocator::template rebind<value_type>::other allocator_type;
|
||||
enum { allow_multimapping = Allow_multimapping };
|
||||
|
||||
concurrent_unordered_map_traits() : my_hash_compare() {}
|
||||
concurrent_unordered_map_traits(const hash_compare& hc) : my_hash_compare(hc) {}
|
||||
|
||||
class value_compare : public std::binary_function<value_type, value_type, bool>
|
||||
{
|
||||
friend class concurrent_unordered_map_traits<Key, T, Hash_compare, Allocator, Allow_multimapping>;
|
||||
|
||||
public:
|
||||
bool operator()(const value_type& left, const value_type& right) const
|
||||
{
|
||||
return (my_hash_compare(left.first, right.first));
|
||||
}
|
||||
|
||||
value_compare(const hash_compare& comparator) : my_hash_compare(comparator) {}
|
||||
|
||||
protected:
|
||||
hash_compare my_hash_compare; // the comparator predicate for keys
|
||||
};
|
||||
|
||||
template<class Type1, class Type2>
|
||||
static const Key& get_key(const std::pair<Type1, Type2>& value) {
|
||||
return (value.first);
|
||||
}
|
||||
|
||||
hash_compare my_hash_compare; // the comparator predicate for keys
|
||||
};
|
||||
|
||||
template <typename Key, typename T, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
|
||||
typename Allocator = tbb::tbb_allocator<std::pair<const Key, T> > >
|
||||
class concurrent_unordered_map :
|
||||
public internal::concurrent_unordered_base< concurrent_unordered_map_traits<Key, T,
|
||||
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> >
|
||||
{
|
||||
// Base type definitions
|
||||
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
|
||||
typedef concurrent_unordered_map_traits<Key, T, hash_compare, Allocator, false> traits_type;
|
||||
typedef internal::concurrent_unordered_base< traits_type > base_type;
|
||||
#if __TBB_EXTRA_DEBUG
|
||||
public:
|
||||
#endif
|
||||
using traits_type::allow_multimapping;
|
||||
public:
|
||||
using base_type::end;
|
||||
using base_type::find;
|
||||
using base_type::insert;
|
||||
|
||||
// Type definitions
|
||||
typedef Key key_type;
|
||||
typedef typename base_type::value_type value_type;
|
||||
typedef T mapped_type;
|
||||
typedef Hasher hasher;
|
||||
typedef Key_equality key_equal;
|
||||
typedef hash_compare key_compare;
|
||||
|
||||
typedef typename base_type::allocator_type allocator_type;
|
||||
typedef typename base_type::pointer pointer;
|
||||
typedef typename base_type::const_pointer const_pointer;
|
||||
typedef typename base_type::reference reference;
|
||||
typedef typename base_type::const_reference const_reference;
|
||||
|
||||
typedef typename base_type::size_type size_type;
|
||||
typedef typename base_type::difference_type difference_type;
|
||||
|
||||
typedef typename base_type::iterator iterator;
|
||||
typedef typename base_type::const_iterator const_iterator;
|
||||
typedef typename base_type::iterator local_iterator;
|
||||
typedef typename base_type::const_iterator const_local_iterator;
|
||||
|
||||
// Construction/destruction/copying
|
||||
explicit concurrent_unordered_map(size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_map(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
|
||||
{
|
||||
}
|
||||
|
||||
template <typename Iterator>
|
||||
concurrent_unordered_map(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
insert(first, last);
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Constructor from initializer_list
|
||||
concurrent_unordered_map(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
this->insert(il.begin(),il.end());
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
concurrent_unordered_map(const concurrent_unordered_map& table)
|
||||
: base_type(table)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_map& operator=(const concurrent_unordered_map& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_map&>(base_type::operator=(table));
|
||||
}
|
||||
|
||||
concurrent_unordered_map(concurrent_unordered_map&& table)
|
||||
: base_type(std::move(table))
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_map& operator=(concurrent_unordered_map&& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_map&>(base_type::operator=(std::move(table)));
|
||||
}
|
||||
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
|
||||
concurrent_unordered_map(const concurrent_unordered_map& table, const Allocator& a)
|
||||
: base_type(table, a)
|
||||
{
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
concurrent_unordered_map(concurrent_unordered_map&& table, const Allocator& a) : base_type(std::move(table), a)
|
||||
{
|
||||
}
|
||||
#endif
|
||||
// Observers
|
||||
mapped_type& operator[](const key_type& key)
|
||||
{
|
||||
iterator where = find(key);
|
||||
|
||||
if (where == end())
|
||||
{
|
||||
where = insert(std::pair<key_type, mapped_type>(key, mapped_type())).first;
|
||||
}
|
||||
|
||||
return ((*where).second);
|
||||
}
|
||||
|
||||
mapped_type& at(const key_type& key)
|
||||
{
|
||||
iterator where = find(key);
|
||||
|
||||
if (where == end())
|
||||
{
|
||||
tbb::internal::throw_exception(tbb::internal::eid_invalid_key);
|
||||
}
|
||||
|
||||
return ((*where).second);
|
||||
}
|
||||
|
||||
const mapped_type& at(const key_type& key) const
|
||||
{
|
||||
const_iterator where = find(key);
|
||||
|
||||
if (where == end())
|
||||
{
|
||||
tbb::internal::throw_exception(tbb::internal::eid_invalid_key);
|
||||
}
|
||||
|
||||
return ((*where).second);
|
||||
}
|
||||
};
|
||||
|
||||
template < typename Key, typename T, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
|
||||
typename Allocator = tbb::tbb_allocator<std::pair<const Key, T> > >
|
||||
class concurrent_unordered_multimap :
|
||||
public internal::concurrent_unordered_base< concurrent_unordered_map_traits< Key, T,
|
||||
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, true> >
|
||||
{
|
||||
// Base type definitions
|
||||
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
|
||||
typedef concurrent_unordered_map_traits<Key, T, hash_compare, Allocator, true> traits_type;
|
||||
typedef internal::concurrent_unordered_base<traits_type> base_type;
|
||||
#if __TBB_EXTRA_DEBUG
|
||||
public:
|
||||
#endif
|
||||
using traits_type::allow_multimapping;
|
||||
public:
|
||||
using base_type::insert;
|
||||
|
||||
// Type definitions
|
||||
typedef Key key_type;
|
||||
typedef typename base_type::value_type value_type;
|
||||
typedef T mapped_type;
|
||||
typedef Hasher hasher;
|
||||
typedef Key_equality key_equal;
|
||||
typedef hash_compare key_compare;
|
||||
|
||||
typedef typename base_type::allocator_type allocator_type;
|
||||
typedef typename base_type::pointer pointer;
|
||||
typedef typename base_type::const_pointer const_pointer;
|
||||
typedef typename base_type::reference reference;
|
||||
typedef typename base_type::const_reference const_reference;
|
||||
|
||||
typedef typename base_type::size_type size_type;
|
||||
typedef typename base_type::difference_type difference_type;
|
||||
|
||||
typedef typename base_type::iterator iterator;
|
||||
typedef typename base_type::const_iterator const_iterator;
|
||||
typedef typename base_type::iterator local_iterator;
|
||||
typedef typename base_type::const_iterator const_local_iterator;
|
||||
|
||||
// Construction/destruction/copying
|
||||
explicit concurrent_unordered_multimap(size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multimap(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
|
||||
{
|
||||
}
|
||||
|
||||
template <typename Iterator>
|
||||
concurrent_unordered_multimap(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets,key_compare(_Hasher,_Key_equality), a)
|
||||
{
|
||||
insert(first, last);
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Constructor from initializer_list
|
||||
concurrent_unordered_multimap(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
this->insert(il.begin(),il.end());
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
concurrent_unordered_multimap(const concurrent_unordered_multimap& table)
|
||||
: base_type(table)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multimap& operator=(const concurrent_unordered_multimap& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_multimap&>(base_type::operator=(table));
|
||||
}
|
||||
|
||||
concurrent_unordered_multimap(concurrent_unordered_multimap&& table)
|
||||
: base_type(std::move(table))
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multimap& operator=(concurrent_unordered_multimap&& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_multimap&>(base_type::operator=(std::move(table)));
|
||||
}
|
||||
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
|
||||
concurrent_unordered_multimap(const concurrent_unordered_multimap& table, const Allocator& a)
|
||||
: base_type(table, a)
|
||||
{
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
concurrent_unordered_multimap(concurrent_unordered_multimap&& table, const Allocator& a) : base_type(std::move(table), a)
|
||||
{
|
||||
}
|
||||
#endif
|
||||
};
|
||||
} // namespace interface5
|
||||
|
||||
using interface5::concurrent_unordered_map;
|
||||
using interface5::concurrent_unordered_multimap;
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif// __TBB_concurrent_unordered_map_H
|
||||
@@ -0,0 +1,269 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
/* Container implementations in this header are based on PPL implementations
|
||||
provided by Microsoft. */
|
||||
|
||||
#ifndef __TBB_concurrent_unordered_set_H
|
||||
#define __TBB_concurrent_unordered_set_H
|
||||
|
||||
#include "internal/_concurrent_unordered_impl.h"
|
||||
|
||||
namespace tbb
|
||||
{
|
||||
|
||||
namespace interface5 {
|
||||
|
||||
// Template class for hash set traits
|
||||
template<typename Key, typename Hash_compare, typename Allocator, bool Allow_multimapping>
|
||||
class concurrent_unordered_set_traits
|
||||
{
|
||||
protected:
|
||||
typedef Key value_type;
|
||||
typedef Key key_type;
|
||||
typedef Hash_compare hash_compare;
|
||||
typedef typename Allocator::template rebind<value_type>::other allocator_type;
|
||||
enum { allow_multimapping = Allow_multimapping };
|
||||
|
||||
concurrent_unordered_set_traits() : my_hash_compare() {}
|
||||
concurrent_unordered_set_traits(const hash_compare& hc) : my_hash_compare(hc) {}
|
||||
|
||||
typedef hash_compare value_compare;
|
||||
|
||||
static const Key& get_key(const value_type& value) {
|
||||
return value;
|
||||
}
|
||||
|
||||
hash_compare my_hash_compare; // the comparator predicate for keys
|
||||
};
|
||||
|
||||
template <typename Key, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>, typename Allocator = tbb::tbb_allocator<Key> >
|
||||
class concurrent_unordered_set : public internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key, internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> >
|
||||
{
|
||||
// Base type definitions
|
||||
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
|
||||
typedef internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key, hash_compare, Allocator, false> > base_type;
|
||||
typedef concurrent_unordered_set_traits<Key, internal::hash_compare<Key, Hasher, Key_equality>, Allocator, false> traits_type;
|
||||
#if __TBB_EXTRA_DEBUG
|
||||
public:
|
||||
#endif
|
||||
using traits_type::allow_multimapping;
|
||||
public:
|
||||
using base_type::insert;
|
||||
|
||||
// Type definitions
|
||||
typedef Key key_type;
|
||||
typedef typename base_type::value_type value_type;
|
||||
typedef Key mapped_type;
|
||||
typedef Hasher hasher;
|
||||
typedef Key_equality key_equal;
|
||||
typedef hash_compare key_compare;
|
||||
|
||||
typedef typename base_type::allocator_type allocator_type;
|
||||
typedef typename base_type::pointer pointer;
|
||||
typedef typename base_type::const_pointer const_pointer;
|
||||
typedef typename base_type::reference reference;
|
||||
typedef typename base_type::const_reference const_reference;
|
||||
|
||||
typedef typename base_type::size_type size_type;
|
||||
typedef typename base_type::difference_type difference_type;
|
||||
|
||||
typedef typename base_type::iterator iterator;
|
||||
typedef typename base_type::const_iterator const_iterator;
|
||||
typedef typename base_type::iterator local_iterator;
|
||||
typedef typename base_type::const_iterator const_local_iterator;
|
||||
|
||||
// Construction/destruction/copying
|
||||
explicit concurrent_unordered_set(size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
|
||||
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_set(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
|
||||
{
|
||||
}
|
||||
|
||||
template <typename Iterator>
|
||||
concurrent_unordered_set(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
|
||||
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
|
||||
{
|
||||
insert(first, last);
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Constructor from initializer_list
|
||||
concurrent_unordered_set(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
|
||||
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
|
||||
{
|
||||
this->insert(il.begin(),il.end());
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
concurrent_unordered_set(const concurrent_unordered_set& table)
|
||||
: base_type(table)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_set& operator=(const concurrent_unordered_set& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_set&>(base_type::operator=(table));
|
||||
}
|
||||
|
||||
concurrent_unordered_set(concurrent_unordered_set&& table)
|
||||
: base_type(std::move(table))
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_set& operator=(concurrent_unordered_set&& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_set&>(base_type::operator=(std::move(table)));
|
||||
}
|
||||
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
|
||||
concurrent_unordered_set(const concurrent_unordered_set& table, const Allocator& a)
|
||||
: base_type(table, a)
|
||||
{
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
concurrent_unordered_set(concurrent_unordered_set&& table, const Allocator& a)
|
||||
: base_type(std::move(table), a)
|
||||
{
|
||||
}
|
||||
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
|
||||
|
||||
};
|
||||
|
||||
template <typename Key, typename Hasher = tbb::tbb_hash<Key>, typename Key_equality = std::equal_to<Key>,
|
||||
typename Allocator = tbb::tbb_allocator<Key> >
|
||||
class concurrent_unordered_multiset :
|
||||
public internal::concurrent_unordered_base< concurrent_unordered_set_traits<Key,
|
||||
internal::hash_compare<Key, Hasher, Key_equality>, Allocator, true> >
|
||||
{
|
||||
// Base type definitions
|
||||
typedef internal::hash_compare<Key, Hasher, Key_equality> hash_compare;
|
||||
typedef concurrent_unordered_set_traits<Key, hash_compare, Allocator, true> traits_type;
|
||||
typedef internal::concurrent_unordered_base< traits_type > base_type;
|
||||
#if __TBB_EXTRA_DEBUG
|
||||
public:
|
||||
#endif
|
||||
using traits_type::allow_multimapping;
|
||||
public:
|
||||
using base_type::insert;
|
||||
|
||||
// Type definitions
|
||||
typedef Key key_type;
|
||||
typedef typename base_type::value_type value_type;
|
||||
typedef Key mapped_type;
|
||||
typedef Hasher hasher;
|
||||
typedef Key_equality key_equal;
|
||||
typedef hash_compare key_compare;
|
||||
|
||||
typedef typename base_type::allocator_type allocator_type;
|
||||
typedef typename base_type::pointer pointer;
|
||||
typedef typename base_type::const_pointer const_pointer;
|
||||
typedef typename base_type::reference reference;
|
||||
typedef typename base_type::const_reference const_reference;
|
||||
|
||||
typedef typename base_type::size_type size_type;
|
||||
typedef typename base_type::difference_type difference_type;
|
||||
|
||||
typedef typename base_type::iterator iterator;
|
||||
typedef typename base_type::const_iterator const_iterator;
|
||||
typedef typename base_type::iterator local_iterator;
|
||||
typedef typename base_type::const_iterator const_local_iterator;
|
||||
|
||||
// Construction/destruction/copying
|
||||
explicit concurrent_unordered_multiset(size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multiset(const Allocator& a) : base_type(base_type::initial_bucket_number, key_compare(), a)
|
||||
{
|
||||
}
|
||||
|
||||
template <typename Iterator>
|
||||
concurrent_unordered_multiset(Iterator first, Iterator last, size_type n_of_buckets = base_type::initial_bucket_number,
|
||||
const hasher& _Hasher = hasher(), const key_equal& _Key_equality = key_equal(),
|
||||
const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(_Hasher, _Key_equality), a)
|
||||
{
|
||||
insert(first, last);
|
||||
}
|
||||
|
||||
#if __TBB_INITIALIZER_LISTS_PRESENT
|
||||
//! Constructor from initializer_list
|
||||
concurrent_unordered_multiset(std::initializer_list<value_type> il, size_type n_of_buckets = base_type::initial_bucket_number, const hasher& a_hasher = hasher(),
|
||||
const key_equal& a_keyeq = key_equal(), const allocator_type& a = allocator_type())
|
||||
: base_type(n_of_buckets, key_compare(a_hasher, a_keyeq), a)
|
||||
{
|
||||
this->insert(il.begin(),il.end());
|
||||
}
|
||||
#endif //# __TBB_INITIALIZER_LISTS_PRESENT
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT && __TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
concurrent_unordered_multiset(const concurrent_unordered_multiset& table)
|
||||
: base_type(table)
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multiset& operator=(const concurrent_unordered_multiset& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_multiset&>(base_type::operator=(table));
|
||||
}
|
||||
|
||||
concurrent_unordered_multiset(concurrent_unordered_multiset&& table)
|
||||
: base_type(std::move(table))
|
||||
{
|
||||
}
|
||||
|
||||
concurrent_unordered_multiset& operator=(concurrent_unordered_multiset&& table)
|
||||
{
|
||||
return static_cast<concurrent_unordered_multiset&>(base_type::operator=(std::move(table)));
|
||||
}
|
||||
#endif //__TBB_CPP11_IMPLICIT_MOVE_MEMBERS_GENERATION_FOR_DERIVED_BROKEN
|
||||
|
||||
concurrent_unordered_multiset(const concurrent_unordered_multiset& table, const Allocator& a)
|
||||
: base_type(table, a)
|
||||
{
|
||||
}
|
||||
|
||||
#if __TBB_CPP11_RVALUE_REF_PRESENT
|
||||
concurrent_unordered_multiset(concurrent_unordered_multiset&& table, const Allocator& a)
|
||||
: base_type(std::move(table), a)
|
||||
{
|
||||
}
|
||||
#endif //__TBB_CPP11_RVALUE_REF_PRESENT
|
||||
};
|
||||
} // namespace interface5
|
||||
|
||||
using interface5::concurrent_unordered_set;
|
||||
using interface5::concurrent_unordered_multiset;
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif// __TBB_concurrent_unordered_set_H
|
||||
@@ -0,0 +1,631 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if (_MSC_VER)
|
||||
//MSVC 10 "deprecated" application of some std:: algorithms to raw pointers as not safe.
|
||||
//The reason is that destination is not checked against bounds/having enough place.
|
||||
#define _SCL_SECURE_NO_WARNINGS
|
||||
#endif
|
||||
|
||||
#include "tbb/concurrent_vector.h"
|
||||
#include "tbb/cache_aligned_allocator.h"
|
||||
#include "tbb/tbb_exception.h"
|
||||
#include "tbb_misc.h"
|
||||
#include "itt_notify.h"
|
||||
|
||||
#if !TBB_USE_EXCEPTIONS && _MSC_VER
|
||||
// Suppress "C++ exception handler used, but unwind semantics are not enabled" warning in STL headers
|
||||
#pragma warning (push)
|
||||
#pragma warning (disable: 4530)
|
||||
#endif
|
||||
|
||||
#include <cstring>
|
||||
#include <memory> //for uninitialized_fill_n
|
||||
|
||||
#if !TBB_USE_EXCEPTIONS && _MSC_VER
|
||||
#pragma warning (pop)
|
||||
#endif
|
||||
|
||||
#if defined(_MSC_VER) && defined(_Wp64)
|
||||
// Workaround for overzealous compiler warnings in /Wp64 mode
|
||||
#pragma warning (disable: 4267)
|
||||
#endif
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
class concurrent_vector_base_v3::helper :no_assign {
|
||||
public:
|
||||
//! memory page size
|
||||
static const size_type page_size = 4096;
|
||||
|
||||
inline static bool incompact_predicate(size_type size) { // assert size != 0, see source/test/test_vector_layout.cpp
|
||||
return size < page_size || ((size-1)%page_size < page_size/2 && size < page_size * 128); // for more details
|
||||
}
|
||||
|
||||
inline static size_type find_segment_end(const concurrent_vector_base_v3 &v) {
|
||||
segment_t *s = v.my_segment;
|
||||
segment_index_t u = s==v.my_storage? pointers_per_short_table : pointers_per_long_table;
|
||||
segment_index_t k = 0;
|
||||
while( k < u && (s[k].load<relaxed>()==segment_allocated() ))
|
||||
++k;
|
||||
return k;
|
||||
}
|
||||
|
||||
// TODO: optimize accesses to my_first_block
|
||||
//! assign first segment size. k - is index of last segment to be allocated, not a count of segments
|
||||
inline static void assign_first_segment_if_necessary(concurrent_vector_base_v3 &v, segment_index_t k) {
|
||||
if( !v.my_first_block ) {
|
||||
/* There was a suggestion to set first segment according to incompact_predicate:
|
||||
while( k && !helper::incompact_predicate(segment_size( k ) * element_size) )
|
||||
--k; // while previous vector size is compact, decrement
|
||||
// reasons to not do it:
|
||||
// * constructor(n) is not ready to accept fragmented segments
|
||||
// * backward compatibility due to that constructor
|
||||
// * current version gives additional guarantee and faster init.
|
||||
// * two calls to reserve() will give the same effect.
|
||||
*/
|
||||
v.my_first_block.compare_and_swap(k+1, 0); // store number of segments
|
||||
}
|
||||
}
|
||||
|
||||
inline static void *allocate_segment(concurrent_vector_base_v3 &v, size_type n) {
|
||||
void *ptr = v.vector_allocator_ptr(v, n);
|
||||
if(!ptr) throw_exception(eid_bad_alloc); // check for bad allocation, throw exception
|
||||
return ptr;
|
||||
}
|
||||
|
||||
//! Publish segment so other threads can see it.
|
||||
template<typename argument_type>
|
||||
inline static void publish_segment( segment_t& s, argument_type rhs ) {
|
||||
// see also itt_store_pointer_with_release_v3()
|
||||
ITT_NOTIFY( sync_releasing, &s );
|
||||
s.store<release>(rhs);
|
||||
}
|
||||
|
||||
static size_type enable_segment(concurrent_vector_base_v3 &v, size_type k, size_type element_size, bool mark_as_not_used_on_failure = false);
|
||||
|
||||
// TODO: rename as get_segments_table() and return segment pointer
|
||||
inline static void extend_table_if_necessary(concurrent_vector_base_v3 &v, size_type k, size_type start ) {
|
||||
if(k >= pointers_per_short_table && v.my_segment == v.my_storage)
|
||||
extend_segment_table(v, start );
|
||||
}
|
||||
|
||||
static void extend_segment_table(concurrent_vector_base_v3 &v, size_type start);
|
||||
|
||||
struct segment_not_used_predicate: no_assign {
|
||||
segment_t &s;
|
||||
segment_not_used_predicate(segment_t &segment) : s(segment) {}
|
||||
bool operator()() const { return s.load<relaxed>() == segment_not_used ();}
|
||||
};
|
||||
inline static segment_t& acquire_segment(concurrent_vector_base_v3 &v, size_type index, size_type element_size, bool owner) {
|
||||
segment_t &s = v.my_segment[index]; // TODO: pass v.my_segment as argument
|
||||
if( s.load<acquire>() == segment_not_used() ) { // do not check for segment_allocation_failed state
|
||||
if( owner ) {
|
||||
enable_segment( v, index, element_size );
|
||||
} else {
|
||||
ITT_NOTIFY(sync_prepare, &s);
|
||||
spin_wait_while(segment_not_used_predicate(s));
|
||||
ITT_NOTIFY(sync_acquired, &s);
|
||||
}
|
||||
} else {
|
||||
ITT_NOTIFY(sync_acquired, &s);
|
||||
}
|
||||
if(s.load<relaxed>() != segment_allocated())
|
||||
throw_exception(eid_bad_last_alloc); // throw custom exception, because it's hard to recover correctly after segment_allocation_failed state
|
||||
return s;
|
||||
}
|
||||
|
||||
///// non-static fields of helper for exception-safe iteration across segments
|
||||
segment_t *table;// TODO: review all segment_index_t as just short type
|
||||
size_type first_block, k, sz, start, finish, element_size;
|
||||
helper(segment_t *segments, size_type fb, size_type esize, size_type index, size_type s, size_type f) throw()
|
||||
: table(segments), first_block(fb), k(index), sz(0), start(s), finish(f), element_size(esize) {}
|
||||
inline void first_segment() throw() {
|
||||
__TBB_ASSERT( start <= finish, NULL );
|
||||
__TBB_ASSERT( first_block || !finish, NULL );
|
||||
if( k < first_block ) k = 0; // process solid segment at a time
|
||||
size_type base = segment_base( k );
|
||||
__TBB_ASSERT( base <= start, NULL );
|
||||
finish -= base; start -= base; // rebase as offsets from segment k
|
||||
sz = k ? base : segment_size( first_block ); // sz==base for k>0
|
||||
}
|
||||
inline void next_segment() throw() {
|
||||
finish -= sz; start = 0; // offsets from next segment
|
||||
if( !k ) k = first_block;
|
||||
else { ++k; sz = segment_size( k ); }
|
||||
}
|
||||
template<typename F>
|
||||
inline size_type apply(const F &func) {
|
||||
first_segment();
|
||||
while( sz < finish ) { // work for more than one segment
|
||||
//TODO: remove extra load() of table[k] inside func
|
||||
func( table[k], table[k].load<relaxed>().pointer<char>() + element_size*start, sz - start );
|
||||
next_segment();
|
||||
}
|
||||
func( table[k], table[k].load<relaxed>().pointer<char>() + element_size*start, finish - start );
|
||||
return k;
|
||||
}
|
||||
inline segment_value_t get_segment_value(size_type index, bool wait) {
|
||||
segment_t &s = table[index];
|
||||
if( wait && (s.load<acquire>() == segment_not_used()) ) {
|
||||
ITT_NOTIFY(sync_prepare, &s);
|
||||
spin_wait_while(segment_not_used_predicate(s));
|
||||
ITT_NOTIFY(sync_acquired, &s);
|
||||
}
|
||||
return s.load<relaxed>();
|
||||
}
|
||||
~helper() {
|
||||
if( sz >= finish ) return; // the work is done correctly
|
||||
cleanup();
|
||||
}
|
||||
|
||||
//! Out of line code to assists destructor in infrequent cases.
|
||||
void cleanup();
|
||||
|
||||
/// TODO: turn into lambda functions when available
|
||||
struct init_body {
|
||||
internal_array_op2 func;
|
||||
const void *arg;
|
||||
init_body(internal_array_op2 init, const void *src) : func(init), arg(src) {}
|
||||
void operator()(segment_t &, void *begin, size_type n) const {
|
||||
func( begin, arg, n );
|
||||
}
|
||||
};
|
||||
struct safe_init_body {
|
||||
internal_array_op2 func;
|
||||
const void *arg;
|
||||
safe_init_body(internal_array_op2 init, const void *src) : func(init), arg(src) {}
|
||||
void operator()(segment_t &s, void *begin, size_type n) const {
|
||||
if(s.load<relaxed>() != segment_allocated())
|
||||
throw_exception(eid_bad_last_alloc); // throw custom exception
|
||||
func( begin, arg, n );
|
||||
}
|
||||
};
|
||||
struct destroy_body {
|
||||
internal_array_op1 func;
|
||||
destroy_body(internal_array_op1 destroy) : func(destroy) {}
|
||||
void operator()(segment_t &s, void *begin, size_type n) const {
|
||||
if(s.load<relaxed>() == segment_allocated())
|
||||
func( begin, n );
|
||||
}
|
||||
};
|
||||
};
|
||||
|
||||
void concurrent_vector_base_v3::helper::extend_segment_table(concurrent_vector_base_v3 &v, concurrent_vector_base_v3::size_type start) {
|
||||
if( start > segment_size(pointers_per_short_table) ) start = segment_size(pointers_per_short_table);
|
||||
// If other threads are trying to set pointers in the short segment, wait for them to finish their
|
||||
// assignments before we copy the short segment to the long segment. Note: grow_to_at_least depends on it
|
||||
for( segment_index_t i = 0; segment_base(i) < start && v.my_segment == v.my_storage; i++ ){
|
||||
if(v.my_storage[i].load<relaxed>() == segment_not_used()) {
|
||||
ITT_NOTIFY(sync_prepare, &v.my_storage[i]);
|
||||
atomic_backoff backoff(true);
|
||||
while( v.my_segment == v.my_storage && (v.my_storage[i].load<relaxed>() == segment_not_used()) )
|
||||
backoff.pause();
|
||||
ITT_NOTIFY(sync_acquired, &v.my_storage[i]);
|
||||
}
|
||||
}
|
||||
if( v.my_segment != v.my_storage ) return;
|
||||
|
||||
segment_t* new_segment_table = (segment_t*)NFS_Allocate( pointers_per_long_table, sizeof(segment_t), NULL );
|
||||
__TBB_ASSERT(new_segment_table, "NFS_Allocate should throws exception if it cannot allocate the requested storage, and not returns zero pointer" );
|
||||
std::uninitialized_fill_n(new_segment_table,size_t(pointers_per_long_table),segment_t()); //init newly allocated table
|
||||
//TODO: replace with static assert
|
||||
__TBB_STATIC_ASSERT(pointers_per_long_table >= pointers_per_short_table, "size of the big table should be not lesser than of the small one, as we copy values to it" );
|
||||
std::copy(v.my_storage, v.my_storage+pointers_per_short_table, new_segment_table);//copy values from old table, here operator= of segment_t is used
|
||||
if( v.my_segment.compare_and_swap( new_segment_table, v.my_storage ) != v.my_storage )
|
||||
NFS_Free( new_segment_table );
|
||||
// else TODO: add ITT_NOTIFY signals for v.my_segment?
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::helper::enable_segment(concurrent_vector_base_v3 &v, concurrent_vector_base_v3::size_type k, concurrent_vector_base_v3::size_type element_size,
|
||||
bool mark_as_not_used_on_failure ) {
|
||||
|
||||
struct segment_scope_guard : no_copy{
|
||||
segment_t* my_segment_ptr;
|
||||
bool my_mark_as_not_used;
|
||||
segment_scope_guard(segment_t& segment, bool mark_as_not_used) : my_segment_ptr(&segment), my_mark_as_not_used(mark_as_not_used){}
|
||||
void dismiss(){ my_segment_ptr = 0;}
|
||||
~segment_scope_guard(){
|
||||
if (my_segment_ptr){
|
||||
if (!my_mark_as_not_used){
|
||||
publish_segment(*my_segment_ptr, segment_allocation_failed());
|
||||
}else{
|
||||
publish_segment(*my_segment_ptr, segment_not_used());
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
segment_t* s = v.my_segment; // TODO: optimize out as argument? Optimize accesses to my_first_block
|
||||
__TBB_ASSERT(s[k].load<relaxed>() != segment_allocated(), "concurrent operation during growth?");
|
||||
|
||||
size_type size_of_enabled_segment = segment_size(k);
|
||||
size_type size_to_allocate = size_of_enabled_segment;
|
||||
if( !k ) {
|
||||
assign_first_segment_if_necessary(v, default_initial_segments-1);
|
||||
size_of_enabled_segment = 2 ;
|
||||
size_to_allocate = segment_size(v.my_first_block);
|
||||
|
||||
} else {
|
||||
spin_wait_while_eq( v.my_first_block, segment_index_t(0) );
|
||||
}
|
||||
|
||||
if( k && (k < v.my_first_block)){ //no need to allocate anything
|
||||
// s[0].array is changed only once ( 0 -> !0 ) and points to uninitialized memory
|
||||
segment_value_t array0 = s[0].load<acquire>();
|
||||
if(array0 == segment_not_used()){
|
||||
// sync_prepare called only if there is a wait
|
||||
ITT_NOTIFY(sync_prepare, &s[0]);
|
||||
spin_wait_while( segment_not_used_predicate(s[0]));
|
||||
array0 = s[0].load<acquire>();
|
||||
}
|
||||
ITT_NOTIFY(sync_acquired, &s[0]);
|
||||
if(array0 != segment_allocated()) { // check for segment_allocation_failed state of initial segment
|
||||
publish_segment(s[k], segment_allocation_failed()); // and assign segment_allocation_failed state here
|
||||
throw_exception(eid_bad_last_alloc); // throw custom exception
|
||||
}
|
||||
publish_segment( s[k],
|
||||
static_cast<void*>(array0.pointer<char>() + segment_base(k)*element_size )
|
||||
);
|
||||
} else {
|
||||
segment_scope_guard k_segment_guard(s[k], mark_as_not_used_on_failure);
|
||||
publish_segment(s[k], allocate_segment(v, size_to_allocate));
|
||||
k_segment_guard.dismiss();
|
||||
}
|
||||
return size_of_enabled_segment;
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::helper::cleanup() {
|
||||
if( !sz ) { // allocation failed, restore the table
|
||||
segment_index_t k_start = k, k_end = segment_index_of(finish-1);
|
||||
if( segment_base( k_start ) < start )
|
||||
get_segment_value(k_start++, true); // wait
|
||||
if( k_start < first_block ) {
|
||||
segment_value_t segment0 = get_segment_value(0, start>0); // wait if necessary
|
||||
if((segment0 != segment_not_used()) && !k_start ) ++k_start;
|
||||
if(segment0 != segment_allocated())
|
||||
for(; k_start < first_block && k_start <= k_end; ++k_start )
|
||||
publish_segment(table[k_start], segment_allocation_failed());
|
||||
else for(; k_start < first_block && k_start <= k_end; ++k_start )
|
||||
publish_segment(table[k_start], static_cast<void*>(
|
||||
(segment0.pointer<char>()) + segment_base(k_start)*element_size) );
|
||||
}
|
||||
for(; k_start <= k_end; ++k_start ) // not in first block
|
||||
if(table[k_start].load<acquire>() == segment_not_used())
|
||||
publish_segment(table[k_start], segment_allocation_failed());
|
||||
// fill allocated items
|
||||
first_segment();
|
||||
goto recover;
|
||||
}
|
||||
while( sz <= finish ) { // there is still work for at least one segment
|
||||
next_segment();
|
||||
recover:
|
||||
segment_value_t array = table[k].load<relaxed>();
|
||||
if(array == segment_allocated())
|
||||
std::memset( (array.pointer<char>()) + element_size*start, 0, ((sz<finish?sz:finish) - start)*element_size );
|
||||
else __TBB_ASSERT( array == segment_allocation_failed(), NULL );
|
||||
}
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::~concurrent_vector_base_v3() {
|
||||
segment_t* s = my_segment;
|
||||
if( s != my_storage ) {
|
||||
#if TBB_USE_ASSERT
|
||||
//to please assert in segment_t destructor
|
||||
std::fill_n(my_storage,size_t(pointers_per_short_table),segment_t());
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
#if TBB_USE_DEBUG
|
||||
for( segment_index_t i = 0; i < pointers_per_long_table; i++)
|
||||
__TBB_ASSERT( my_segment[i].load<relaxed>() != segment_allocated(), "Segment should have been freed. Please recompile with new TBB before using exceptions.");
|
||||
#endif
|
||||
my_segment = my_storage;
|
||||
NFS_Free( s );
|
||||
}
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_capacity() const {
|
||||
return segment_base( helper::find_segment_end(*this) );
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_throw_exception(size_type t) const {
|
||||
switch(t) {
|
||||
case 0: throw_exception(eid_out_of_range);
|
||||
case 1: throw_exception(eid_segment_range_error);
|
||||
case 2: throw_exception(eid_index_range_error);
|
||||
}
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_reserve( size_type n, size_type element_size, size_type max_size ) {
|
||||
if( n>max_size )
|
||||
throw_exception(eid_reservation_length_error);
|
||||
__TBB_ASSERT( n, NULL );
|
||||
helper::assign_first_segment_if_necessary(*this, segment_index_of(n-1));
|
||||
segment_index_t k = helper::find_segment_end(*this);
|
||||
|
||||
for( ; segment_base(k)<n; ++k ) {
|
||||
helper::extend_table_if_necessary(*this, k, 0);
|
||||
if(my_segment[k].load<relaxed>() != segment_allocated())
|
||||
helper::enable_segment(*this, k, element_size, true ); //in case of failure mark segments as not used
|
||||
}
|
||||
}
|
||||
|
||||
//TODO: Looks like atomic loads can be done relaxed here, as the only place this method is called from
|
||||
//is the constructor, which does not require synchronization (for more details see comment in the
|
||||
// concurrent_vector_base constructor).
|
||||
void concurrent_vector_base_v3::internal_copy( const concurrent_vector_base_v3& src, size_type element_size, internal_array_op2 copy ) {
|
||||
size_type n = src.my_early_size;
|
||||
__TBB_ASSERT( my_segment == my_storage, NULL);
|
||||
if( n ) {
|
||||
helper::assign_first_segment_if_necessary(*this, segment_index_of(n-1));
|
||||
size_type b;
|
||||
for( segment_index_t k=0; (b=segment_base(k))<n; ++k ) {
|
||||
if( (src.my_segment.load<acquire>() == src.my_storage && k >= pointers_per_short_table)
|
||||
|| (src.my_segment[k].load<relaxed>() != segment_allocated())) {
|
||||
my_early_size = b; break;
|
||||
}
|
||||
helper::extend_table_if_necessary(*this, k, 0);
|
||||
size_type m = helper::enable_segment(*this, k, element_size);
|
||||
if( m > n-b ) m = n-b;
|
||||
my_early_size = b+m;
|
||||
copy( my_segment[k].load<relaxed>().pointer<void>(), src.my_segment[k].load<relaxed>().pointer<void>(), m );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_assign( const concurrent_vector_base_v3& src, size_type element_size, internal_array_op1 destroy, internal_array_op2 assign, internal_array_op2 copy ) {
|
||||
size_type n = src.my_early_size;
|
||||
while( my_early_size>n ) { // TODO: improve
|
||||
segment_index_t k = segment_index_of( my_early_size-1 );
|
||||
size_type b=segment_base(k);
|
||||
size_type new_end = b>=n ? b : n;
|
||||
__TBB_ASSERT( my_early_size>new_end, NULL );
|
||||
if( my_segment[k].load<relaxed>() != segment_allocated()) // check vector was broken before
|
||||
throw_exception(eid_bad_last_alloc); // throw custom exception
|
||||
// destructors are supposed to not throw any exceptions
|
||||
destroy( my_segment[k].load<relaxed>().pointer<char>() + element_size*(new_end-b), my_early_size-new_end );
|
||||
my_early_size = new_end;
|
||||
}
|
||||
size_type dst_initialized_size = my_early_size;
|
||||
my_early_size = n;
|
||||
helper::assign_first_segment_if_necessary(*this, segment_index_of(n));
|
||||
size_type b;
|
||||
for( segment_index_t k=0; (b=segment_base(k))<n; ++k ) {
|
||||
if( (src.my_segment.load<acquire>() == src.my_storage && k >= pointers_per_short_table)
|
||||
|| src.my_segment[k].load<relaxed>() != segment_allocated() ) { // if source is damaged
|
||||
my_early_size = b; break; // TODO: it may cause undestructed items
|
||||
}
|
||||
helper::extend_table_if_necessary(*this, k, 0);
|
||||
if( my_segment[k].load<relaxed>() == segment_not_used())
|
||||
helper::enable_segment(*this, k, element_size);
|
||||
else if( my_segment[k].load<relaxed>() != segment_allocated() )
|
||||
throw_exception(eid_bad_last_alloc); // throw custom exception
|
||||
size_type m = k? segment_size(k) : 2;
|
||||
if( m > n-b ) m = n-b;
|
||||
size_type a = 0;
|
||||
if( dst_initialized_size>b ) {
|
||||
a = dst_initialized_size-b;
|
||||
if( a>m ) a = m;
|
||||
assign( my_segment[k].load<relaxed>().pointer<void>(), src.my_segment[k].load<relaxed>().pointer<void>(), a );
|
||||
m -= a;
|
||||
a *= element_size;
|
||||
}
|
||||
if( m>0 )
|
||||
copy( my_segment[k].load<relaxed>().pointer<char>() + a, src.my_segment[k].load<relaxed>().pointer<char>() + a, m );
|
||||
}
|
||||
__TBB_ASSERT( src.my_early_size==n, "detected use of concurrent_vector::operator= with right side that was concurrently modified" );
|
||||
}
|
||||
|
||||
void* concurrent_vector_base_v3::internal_push_back( size_type element_size, size_type& index ) {
|
||||
__TBB_ASSERT( sizeof(my_early_size)==sizeof(uintptr_t), NULL );
|
||||
size_type tmp = my_early_size.fetch_and_increment<acquire>();
|
||||
index = tmp;
|
||||
segment_index_t k_old = segment_index_of( tmp );
|
||||
size_type base = segment_base(k_old);
|
||||
helper::extend_table_if_necessary(*this, k_old, tmp);
|
||||
segment_t& s = helper::acquire_segment(*this, k_old, element_size, base==tmp);
|
||||
size_type j_begin = tmp-base;
|
||||
return (void*)(s.load<relaxed>().pointer<char>() + element_size*j_begin);
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_grow_to_at_least( size_type new_size, size_type element_size, internal_array_op2 init, const void *src ) {
|
||||
internal_grow_to_at_least_with_result( new_size, element_size, init, src );
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_grow_to_at_least_with_result( size_type new_size, size_type element_size, internal_array_op2 init, const void *src ) {
|
||||
size_type e = my_early_size;
|
||||
while( e<new_size ) {
|
||||
size_type f = my_early_size.compare_and_swap(new_size,e);
|
||||
if( f==e ) {
|
||||
internal_grow( e, new_size, element_size, init, src );
|
||||
break;
|
||||
}
|
||||
e = f;
|
||||
}
|
||||
// Check/wait for segments allocation completes
|
||||
segment_index_t i, k_old = segment_index_of( new_size-1 );
|
||||
if( k_old >= pointers_per_short_table && my_segment == my_storage ) {
|
||||
spin_wait_while_eq( my_segment, my_storage );
|
||||
}
|
||||
for( i = 0; i <= k_old; ++i ) {
|
||||
segment_t &s = my_segment[i];
|
||||
if(s.load<relaxed>() == segment_not_used()) {
|
||||
ITT_NOTIFY(sync_prepare, &s);
|
||||
atomic_backoff backoff(true);
|
||||
while( my_segment[i].load<acquire>() == segment_not_used() ) // my_segment may change concurrently
|
||||
backoff.pause();
|
||||
ITT_NOTIFY(sync_acquired, &s);
|
||||
}
|
||||
if( my_segment[i].load<relaxed>() != segment_allocated() )
|
||||
throw_exception(eid_bad_last_alloc);
|
||||
}
|
||||
#if TBB_USE_DEBUG
|
||||
size_type capacity = internal_capacity();
|
||||
__TBB_ASSERT( capacity >= new_size, NULL);
|
||||
#endif
|
||||
return e;
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::size_type concurrent_vector_base_v3::internal_grow_by( size_type delta, size_type element_size, internal_array_op2 init, const void *src ) {
|
||||
size_type result = my_early_size.fetch_and_add(delta);
|
||||
internal_grow( result, result+delta, element_size, init, src );
|
||||
return result;
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_grow( const size_type start, size_type finish, size_type element_size, internal_array_op2 init, const void *src ) {
|
||||
__TBB_ASSERT( start<finish, "start must be less than finish" );
|
||||
segment_index_t k_start = segment_index_of(start), k_end = segment_index_of(finish-1);
|
||||
helper::assign_first_segment_if_necessary(*this, k_end);
|
||||
helper::extend_table_if_necessary(*this, k_end, start);
|
||||
helper range(my_segment, my_first_block, element_size, k_start, start, finish);
|
||||
for(; k_end > k_start && k_end >= range.first_block; --k_end ) // allocate segments in reverse order
|
||||
helper::acquire_segment(*this, k_end, element_size, true/*for k_end>k_start*/);
|
||||
for(; k_start <= k_end; ++k_start ) // but allocate first block in straight order
|
||||
helper::acquire_segment(*this, k_start, element_size, segment_base( k_start ) >= start );
|
||||
range.apply( helper::init_body(init, src) );
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_resize( size_type n, size_type element_size, size_type max_size, const void *src,
|
||||
internal_array_op1 destroy, internal_array_op2 init ) {
|
||||
size_type j = my_early_size;
|
||||
if( n > j ) { // construct items
|
||||
internal_reserve(n, element_size, max_size);
|
||||
my_early_size = n;
|
||||
helper for_each(my_segment, my_first_block, element_size, segment_index_of(j), j, n);
|
||||
for_each.apply( helper::safe_init_body(init, src) );
|
||||
} else {
|
||||
my_early_size = n;
|
||||
helper for_each(my_segment, my_first_block, element_size, segment_index_of(n), n, j);
|
||||
for_each.apply( helper::destroy_body(destroy) );
|
||||
}
|
||||
}
|
||||
|
||||
concurrent_vector_base_v3::segment_index_t concurrent_vector_base_v3::internal_clear( internal_array_op1 destroy ) {
|
||||
__TBB_ASSERT( my_segment, NULL );
|
||||
size_type j = my_early_size;
|
||||
my_early_size = 0;
|
||||
helper for_each(my_segment, my_first_block, 0, 0, 0, j); // element_size is safe to be zero if 'start' is zero
|
||||
j = for_each.apply( helper::destroy_body(destroy) );
|
||||
size_type i = helper::find_segment_end(*this);
|
||||
return j < i? i : j+1;
|
||||
}
|
||||
|
||||
void *concurrent_vector_base_v3::internal_compact( size_type element_size, void *table, internal_array_op1 destroy, internal_array_op2 copy )
|
||||
{
|
||||
const size_type my_size = my_early_size;
|
||||
const segment_index_t k_end = helper::find_segment_end(*this); // allocated segments
|
||||
const segment_index_t k_stop = my_size? segment_index_of(my_size-1) + 1 : 0; // number of segments to store existing items: 0=>0; 1,2=>1; 3,4=>2; [5-8]=>3;..
|
||||
const segment_index_t first_block = my_first_block; // number of merged segments, getting values from atomics
|
||||
|
||||
segment_index_t k = first_block;
|
||||
if(k_stop < first_block)
|
||||
k = k_stop;
|
||||
else
|
||||
while (k < k_stop && helper::incompact_predicate(segment_size( k ) * element_size) ) k++;
|
||||
if(k_stop == k_end && k == first_block)
|
||||
return NULL;
|
||||
|
||||
segment_t *const segment_table = my_segment;
|
||||
internal_segments_table &old = *static_cast<internal_segments_table*>( table );
|
||||
//this call is left here for sake of backward compatibility, and as a placeholder for table initialization
|
||||
std::fill_n(old.table,sizeof(old.table)/sizeof(old.table[0]),segment_t());
|
||||
old.first_block=0;
|
||||
|
||||
if ( k != first_block && k ) // first segment optimization
|
||||
{
|
||||
// exception can occur here
|
||||
void *seg = helper::allocate_segment(*this, segment_size(k));
|
||||
old.table[0].store<relaxed>(seg);
|
||||
old.first_block = k; // fill info for freeing new segment if exception occurs
|
||||
// copy items to the new segment
|
||||
size_type my_segment_size = segment_size( first_block );
|
||||
for (segment_index_t i = 0, j = 0; i < k && j < my_size; j = my_segment_size) {
|
||||
__TBB_ASSERT( segment_table[i].load<relaxed>() == segment_allocated(), NULL);
|
||||
void *s = static_cast<void*>(
|
||||
static_cast<char*>(seg) + segment_base(i)*element_size );
|
||||
//TODO: refactor to use std::min
|
||||
if(j + my_segment_size >= my_size) my_segment_size = my_size - j;
|
||||
__TBB_TRY { // exception can occur here
|
||||
copy( s, segment_table[i].load<relaxed>().pointer<void>(), my_segment_size );
|
||||
} __TBB_CATCH(...) { // destroy all the already copied items
|
||||
helper for_each(&old.table[0], old.first_block, element_size,
|
||||
0, 0, segment_base(i)+ my_segment_size);
|
||||
for_each.apply( helper::destroy_body(destroy) );
|
||||
__TBB_RETHROW();
|
||||
}
|
||||
my_segment_size = i? segment_size( ++i ) : segment_size( i = first_block );
|
||||
}
|
||||
// commit the changes
|
||||
std::copy(segment_table,segment_table + k,old.table);
|
||||
for (segment_index_t i = 0; i < k; i++) {
|
||||
segment_table[i].store<relaxed>(static_cast<void*>(
|
||||
static_cast<char*>(seg) + segment_base(i)*element_size ));
|
||||
}
|
||||
old.first_block = first_block; my_first_block = k; // now, first_block != my_first_block
|
||||
// destroy original copies
|
||||
my_segment_size = segment_size( first_block ); // old.first_block actually
|
||||
for (segment_index_t i = 0, j = 0; i < k && j < my_size; j = my_segment_size) {
|
||||
if(j + my_segment_size >= my_size) my_segment_size = my_size - j;
|
||||
// destructors are supposed to not throw any exceptions
|
||||
destroy( old.table[i].load<relaxed>().pointer<void>(), my_segment_size );
|
||||
my_segment_size = i? segment_size( ++i ) : segment_size( i = first_block );
|
||||
}
|
||||
}
|
||||
// free unnecessary segments allocated by reserve() call
|
||||
if ( k_stop < k_end ) {
|
||||
old.first_block = first_block;
|
||||
std::copy(segment_table+k_stop, segment_table+k_end, old.table+k_stop );
|
||||
std::fill_n(segment_table+k_stop, (k_end-k_stop), segment_t());
|
||||
if( !k ) my_first_block = 0;
|
||||
}
|
||||
return table;
|
||||
}
|
||||
|
||||
void concurrent_vector_base_v3::internal_swap(concurrent_vector_base_v3& v)
|
||||
{
|
||||
size_type my_sz = my_early_size.load<acquire>();
|
||||
size_type v_sz = v.my_early_size.load<relaxed>();
|
||||
if(!my_sz && !v_sz) return;
|
||||
|
||||
bool my_was_short = (my_segment.load<relaxed>() == my_storage);
|
||||
bool v_was_short = (v.my_segment.load<relaxed>() == v.my_storage);
|
||||
|
||||
//In C++11, this would be: swap(my_storage, v.my_storage);
|
||||
for (int i=0; i < pointers_per_short_table; ++i){
|
||||
swap(my_storage[i], v.my_storage[i]);
|
||||
}
|
||||
tbb::internal::swap<relaxed>(my_first_block, v.my_first_block);
|
||||
tbb::internal::swap<relaxed>(my_segment, v.my_segment);
|
||||
if (my_was_short){
|
||||
v.my_segment.store<relaxed>(v.my_storage);
|
||||
}
|
||||
if(v_was_short){
|
||||
my_segment.store<relaxed>(my_storage);
|
||||
}
|
||||
|
||||
my_early_size.store<relaxed>(v_sz);
|
||||
v.my_early_size.store<release>(my_sz);
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
|
||||
} // tbb
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,199 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
#include "tbb/compat/condition_variable"
|
||||
#include "tbb/atomic.h"
|
||||
#include "tbb_misc.h"
|
||||
#include "dynamic_link.h"
|
||||
#include "itt_notify.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
|
||||
//condition_variable
|
||||
#if _WIN32||_WIN64
|
||||
using tbb::interface5::internal::condition_variable_using_event;
|
||||
|
||||
static atomic<do_once_state> condvar_api_state;
|
||||
|
||||
void WINAPI init_condvar_using_event( condition_variable_using_event* cv_event )
|
||||
{
|
||||
// TODO: For Metro port, we can always use the API for condition variables, without dynamic_link etc.
|
||||
cv_event->event = CreateEventEx(NULL, NULL, 0x1 /*CREATE_EVENT_MANUAL_RESET*/, EVENT_ALL_ACCESS );
|
||||
InitializeCriticalSectionEx( &cv_event->mutex, 4000, 0 );
|
||||
cv_event->n_waiters = 0;
|
||||
cv_event->release_count = 0;
|
||||
cv_event->epoch = 0;
|
||||
}
|
||||
|
||||
BOOL WINAPI sleep_condition_variable_cs_using_event( condition_variable_using_event* cv_event, LPCRITICAL_SECTION cs, DWORD dwMilliseconds )
|
||||
{
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
++cv_event->n_waiters;
|
||||
unsigned my_generation = cv_event->epoch;
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
LeaveCriticalSection( cs );
|
||||
for (;;) {
|
||||
// should come here at least once
|
||||
DWORD rc = WaitForSingleObjectEx( cv_event->event, dwMilliseconds, FALSE );
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
if( rc!=WAIT_OBJECT_0 ) {
|
||||
--cv_event->n_waiters;
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
if( rc==WAIT_TIMEOUT ) {
|
||||
SetLastError( WAIT_TIMEOUT );
|
||||
EnterCriticalSection( cs );
|
||||
}
|
||||
return false;
|
||||
}
|
||||
__TBB_ASSERT( rc==WAIT_OBJECT_0, NULL );
|
||||
if( cv_event->release_count>0 && cv_event->epoch!=my_generation )
|
||||
break;
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
}
|
||||
|
||||
// still in the critical section
|
||||
--cv_event->n_waiters;
|
||||
int count = --cv_event->release_count;
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
|
||||
if( count==0 ) {
|
||||
__TBB_ASSERT( cv_event->event, "Premature destruction of condition variable?" );
|
||||
ResetEvent( cv_event->event );
|
||||
}
|
||||
EnterCriticalSection( cs );
|
||||
return true;
|
||||
}
|
||||
|
||||
void WINAPI wake_condition_variable_using_event( condition_variable_using_event* cv_event )
|
||||
{
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
if( cv_event->n_waiters>cv_event->release_count ) {
|
||||
SetEvent( cv_event->event ); // Signal the manual-reset event.
|
||||
++cv_event->release_count;
|
||||
++cv_event->epoch;
|
||||
}
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
}
|
||||
|
||||
void WINAPI wake_all_condition_variable_using_event( condition_variable_using_event* cv_event )
|
||||
{
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
if( cv_event->n_waiters>0 ) {
|
||||
SetEvent( cv_event->event );
|
||||
cv_event->release_count = cv_event->n_waiters;
|
||||
++cv_event->epoch;
|
||||
}
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
}
|
||||
|
||||
void WINAPI destroy_condvar_using_event( condition_variable_using_event* cv_event )
|
||||
{
|
||||
HANDLE my_event = cv_event->event;
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
// NULL is an invalid HANDLE value
|
||||
cv_event->event = NULL;
|
||||
if( cv_event->n_waiters>0 ) {
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
spin_wait_until_eq( cv_event->n_waiters, 0 );
|
||||
// make sure the last thread completes its access to cv
|
||||
EnterCriticalSection( &cv_event->mutex );
|
||||
}
|
||||
LeaveCriticalSection( &cv_event->mutex );
|
||||
CloseHandle( my_event );
|
||||
}
|
||||
|
||||
void WINAPI destroy_condvar_noop( CONDITION_VARIABLE* /*cv*/ ) { /*no op*/ }
|
||||
|
||||
static void (WINAPI *__TBB_init_condvar)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&init_condvar_using_event;
|
||||
static BOOL (WINAPI *__TBB_condvar_wait)( PCONDITION_VARIABLE, LPCRITICAL_SECTION, DWORD ) = (BOOL (WINAPI *)(PCONDITION_VARIABLE,LPCRITICAL_SECTION, DWORD))&sleep_condition_variable_cs_using_event;
|
||||
static void (WINAPI *__TBB_condvar_notify_one)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&wake_condition_variable_using_event;
|
||||
static void (WINAPI *__TBB_condvar_notify_all)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&wake_all_condition_variable_using_event;
|
||||
static void (WINAPI *__TBB_destroy_condvar)( PCONDITION_VARIABLE ) = (void (WINAPI *)(PCONDITION_VARIABLE))&destroy_condvar_using_event;
|
||||
|
||||
//! Table describing how to link the handlers.
|
||||
static const dynamic_link_descriptor CondVarLinkTable[] = {
|
||||
DLD(InitializeConditionVariable, __TBB_init_condvar),
|
||||
DLD(SleepConditionVariableCS, __TBB_condvar_wait),
|
||||
DLD(WakeConditionVariable, __TBB_condvar_notify_one),
|
||||
DLD(WakeAllConditionVariable, __TBB_condvar_notify_all)
|
||||
};
|
||||
|
||||
void init_condvar_module()
|
||||
{
|
||||
__TBB_ASSERT( (uintptr_t)__TBB_init_condvar==(uintptr_t)&init_condvar_using_event, NULL );
|
||||
if( dynamic_link( "Kernel32.dll", CondVarLinkTable, 4 ) )
|
||||
__TBB_destroy_condvar = (void (WINAPI *)(PCONDITION_VARIABLE))&destroy_condvar_noop;
|
||||
}
|
||||
#endif /* _WIN32||_WIN64 */
|
||||
|
||||
} // namespace internal
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
|
||||
namespace interface5 {
|
||||
namespace internal {
|
||||
|
||||
using tbb::internal::condvar_api_state;
|
||||
using tbb::internal::__TBB_init_condvar;
|
||||
using tbb::internal::__TBB_condvar_wait;
|
||||
using tbb::internal::__TBB_condvar_notify_one;
|
||||
using tbb::internal::__TBB_condvar_notify_all;
|
||||
using tbb::internal::__TBB_destroy_condvar;
|
||||
using tbb::internal::init_condvar_module;
|
||||
|
||||
void internal_initialize_condition_variable( condvar_impl_t& cv )
|
||||
{
|
||||
atomic_do_once( &init_condvar_module, condvar_api_state );
|
||||
__TBB_init_condvar( &cv.cv_native );
|
||||
}
|
||||
|
||||
void internal_destroy_condition_variable( condvar_impl_t& cv )
|
||||
{
|
||||
__TBB_destroy_condvar( &cv.cv_native );
|
||||
}
|
||||
|
||||
void internal_condition_variable_notify_one( condvar_impl_t& cv )
|
||||
{
|
||||
__TBB_condvar_notify_one ( &cv.cv_native );
|
||||
}
|
||||
|
||||
void internal_condition_variable_notify_all( condvar_impl_t& cv )
|
||||
{
|
||||
__TBB_condvar_notify_all( &cv.cv_native );
|
||||
}
|
||||
|
||||
bool internal_condition_variable_wait( condvar_impl_t& cv, mutex* mtx, const tick_count::interval_t* i )
|
||||
{
|
||||
DWORD duration = i ? DWORD((i->seconds()*1000)) : INFINITE;
|
||||
mtx->set_state( mutex::INITIALIZED );
|
||||
BOOL res = __TBB_condvar_wait( &cv.cv_native, mtx->native_handle(), duration );
|
||||
mtx->set_state( mutex::HELD );
|
||||
return res?true:false;
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
} // nameespace interface5
|
||||
|
||||
#endif /* _WIN32||_WIN64 */
|
||||
|
||||
} // namespace tbb
|
||||
@@ -0,0 +1,31 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/critical_section.h"
|
||||
#include "itt_notify.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
void critical_section_v4::internal_construct() {
|
||||
ITT_SYNC_CREATE(&my_impl, _T("ppl::critical_section"), _T(""));
|
||||
}
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_CRITICAL_SECTION_H_
|
||||
#define _TBB_CRITICAL_SECTION_H_
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#include "machine/windows_api.h"
|
||||
#else
|
||||
#include <pthread.h>
|
||||
#include <errno.h>
|
||||
#endif // _WIN32||WIN64
|
||||
|
||||
#include "tbb_stddef.h"
|
||||
#include "tbb_thread.h"
|
||||
#include "tbb_exception.h"
|
||||
|
||||
#include "tbb_profiling.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
class critical_section_v4 : internal::no_copy {
|
||||
#if _WIN32||_WIN64
|
||||
CRITICAL_SECTION my_impl;
|
||||
#else
|
||||
pthread_mutex_t my_impl;
|
||||
#endif
|
||||
tbb_thread::id my_tid;
|
||||
public:
|
||||
|
||||
void __TBB_EXPORTED_METHOD internal_construct();
|
||||
|
||||
critical_section_v4() {
|
||||
#if _WIN32||_WIN64
|
||||
InitializeCriticalSectionEx( &my_impl, 4000, 0 );
|
||||
#else
|
||||
pthread_mutex_init(&my_impl, NULL);
|
||||
#endif
|
||||
internal_construct();
|
||||
}
|
||||
|
||||
~critical_section_v4() {
|
||||
__TBB_ASSERT(my_tid == tbb_thread::id(), "Destroying a still-held critical section");
|
||||
#if _WIN32||_WIN64
|
||||
DeleteCriticalSection(&my_impl);
|
||||
#else
|
||||
pthread_mutex_destroy(&my_impl);
|
||||
#endif
|
||||
}
|
||||
|
||||
class scoped_lock : internal::no_copy {
|
||||
private:
|
||||
critical_section_v4 &my_crit;
|
||||
public:
|
||||
scoped_lock( critical_section_v4& lock_me) :my_crit(lock_me) {
|
||||
my_crit.lock();
|
||||
}
|
||||
|
||||
~scoped_lock() {
|
||||
my_crit.unlock();
|
||||
}
|
||||
};
|
||||
|
||||
void lock() {
|
||||
tbb_thread::id local_tid = this_tbb_thread::get_id();
|
||||
if(local_tid == my_tid) throw_exception( eid_improper_lock );
|
||||
#if _WIN32||_WIN64
|
||||
EnterCriticalSection( &my_impl );
|
||||
#else
|
||||
int rval = pthread_mutex_lock(&my_impl);
|
||||
__TBB_ASSERT_EX(!rval, "critical_section::lock: pthread_mutex_lock failed");
|
||||
#endif
|
||||
__TBB_ASSERT(my_tid == tbb_thread::id(), NULL);
|
||||
my_tid = local_tid;
|
||||
}
|
||||
|
||||
bool try_lock() {
|
||||
bool gotlock;
|
||||
tbb_thread::id local_tid = this_tbb_thread::get_id();
|
||||
if(local_tid == my_tid) return false;
|
||||
#if _WIN32||_WIN64
|
||||
gotlock = TryEnterCriticalSection( &my_impl ) != 0;
|
||||
#else
|
||||
int rval = pthread_mutex_trylock(&my_impl);
|
||||
// valid returns are 0 (locked) and [EBUSY]
|
||||
__TBB_ASSERT(rval == 0 || rval == EBUSY, "critical_section::trylock: pthread_mutex_trylock failed");
|
||||
gotlock = rval == 0;
|
||||
#endif
|
||||
if(gotlock) {
|
||||
my_tid = local_tid;
|
||||
}
|
||||
return gotlock;
|
||||
}
|
||||
|
||||
void unlock() {
|
||||
__TBB_ASSERT(this_tbb_thread::get_id() == my_tid, "thread unlocking critical_section is not thread that locked it");
|
||||
my_tid = tbb_thread::id();
|
||||
#if _WIN32||_WIN64
|
||||
LeaveCriticalSection( &my_impl );
|
||||
#else
|
||||
int rval = pthread_mutex_unlock(&my_impl);
|
||||
__TBB_ASSERT_EX(!rval, "critical_section::unlock: pthread_mutex_unlock failed");
|
||||
#endif
|
||||
}
|
||||
|
||||
static const bool is_rw_mutex = false;
|
||||
static const bool is_recursive_mutex = false;
|
||||
static const bool is_fair_mutex = true;
|
||||
}; // critical_section_v4
|
||||
} // namespace internal
|
||||
typedef internal::critical_section_v4 critical_section;
|
||||
|
||||
__TBB_DEFINE_PROFILING_SET_NAME(critical_section)
|
||||
} // namespace tbb
|
||||
#endif // _TBB_CRITICAL_SECTION_H_
|
||||
@@ -0,0 +1,684 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_custom_scheduler_H
|
||||
#define _TBB_custom_scheduler_H
|
||||
|
||||
#include "scheduler.h"
|
||||
#include "observer_proxy.h"
|
||||
#include "itt_notify.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
//! Amount of time to pause between steals.
|
||||
/** The default values below were found to be best empirically for K-Means
|
||||
on the 32-way Altix and 4-way (*2 for HT) fxqlin04. */
|
||||
#ifdef __TBB_STEALING_PAUSE
|
||||
static const long PauseTime = __TBB_STEALING_PAUSE;
|
||||
#elif __TBB_ipf
|
||||
static const long PauseTime = 1500;
|
||||
#else
|
||||
static const long PauseTime = 80;
|
||||
#endif
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
//! Traits classes for scheduler
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
struct DefaultSchedulerTraits {
|
||||
static const bool itt_possible = true;
|
||||
static const bool has_slow_atomic = false;
|
||||
};
|
||||
|
||||
struct IntelSchedulerTraits {
|
||||
static const bool itt_possible = false;
|
||||
#if __TBB_x86_32||__TBB_x86_64
|
||||
static const bool has_slow_atomic = true;
|
||||
#else
|
||||
static const bool has_slow_atomic = false;
|
||||
#endif /* __TBB_x86_32||__TBB_x86_64 */
|
||||
};
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// custom_scheduler
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
//! A scheduler with a customized evaluation loop.
|
||||
/** The customization can use SchedulerTraits to make decisions without needing a run-time check. */
|
||||
template<typename SchedulerTraits>
|
||||
class custom_scheduler: private generic_scheduler {
|
||||
typedef custom_scheduler<SchedulerTraits> scheduler_type;
|
||||
|
||||
//! Scheduler loop that dispatches tasks.
|
||||
/** If child is non-NULL, it is dispatched first.
|
||||
Then, until "parent" has a reference count of 1, other task are dispatched or stolen. */
|
||||
/*override*/
|
||||
void local_wait_for_all( task& parent, task* child );
|
||||
|
||||
//! Entry point from client code to the scheduler loop that dispatches tasks.
|
||||
/** The method is virtual, but the *this object is used only for sake of dispatching on the correct vtable,
|
||||
not necessarily the correct *this object. The correct *this object is looked up in TLS. */
|
||||
/*override*/
|
||||
void wait_for_all( task& parent, task* child ) {
|
||||
static_cast<custom_scheduler*>(governor::local_scheduler())->scheduler_type::local_wait_for_all( parent, child );
|
||||
}
|
||||
|
||||
//! Construct a custom_scheduler
|
||||
custom_scheduler( arena* a, size_t index ) : generic_scheduler(a, index) {}
|
||||
|
||||
//! Decrements ref_count of a predecessor.
|
||||
/** If it achieves 0, the predecessor is scheduled for execution.
|
||||
When changing, remember that this is a hot path function. */
|
||||
void tally_completion_of_predecessor( task& s, task*& bypass_slot ) {
|
||||
task_prefix& p = s.prefix();
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_releasing, &p.ref_count);
|
||||
if( SchedulerTraits::has_slow_atomic && p.ref_count==1 )
|
||||
p.ref_count=0;
|
||||
else if( __TBB_FetchAndDecrementWrelease(&p.ref_count) > 1 ) {// more references exist
|
||||
// '__TBB_cl_evict(&p)' degraded performance of parallel_preorder example
|
||||
return;
|
||||
}
|
||||
|
||||
// Ordering on p.ref_count (superfluous if SchedulerTraits::has_slow_atomic)
|
||||
__TBB_control_consistency_helper();
|
||||
__TBB_ASSERT(p.ref_count==0, "completion of task caused predecessor's reference count to underflow");
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_acquired, &p.ref_count);
|
||||
#if TBB_USE_ASSERT
|
||||
p.extra_state &= ~es_ref_count_active;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
|
||||
#if __TBB_RECYCLE_TO_ENQUEUE
|
||||
if (p.state==task::to_enqueue) {
|
||||
// related to __TBB_TASK_ARENA TODO: try keep priority of the task
|
||||
// e.g. rework task_prefix to remember priority of received task and use here
|
||||
my_arena->enqueue_task(s, 0, my_random );
|
||||
} else
|
||||
#endif /*__TBB_RECYCLE_TO_ENQUEUE*/
|
||||
if( bypass_slot==NULL )
|
||||
bypass_slot = &s;
|
||||
else
|
||||
local_spawn( s, s.prefix().next );
|
||||
}
|
||||
|
||||
public:
|
||||
static generic_scheduler* allocate_scheduler( arena* a, size_t index ) {
|
||||
scheduler_type* s = (scheduler_type*)NFS_Allocate(1,sizeof(scheduler_type),NULL);
|
||||
new( s ) scheduler_type( a, index );
|
||||
s->assert_task_pool_valid();
|
||||
ITT_SYNC_CREATE(s, SyncType_Scheduler, SyncObj_TaskPoolSpinning);
|
||||
return s;
|
||||
}
|
||||
|
||||
//! Try getting a task from the mailbox or stealing from another scheduler.
|
||||
/** Returns the stolen task or NULL if all attempts fail. */
|
||||
/* override */ task* receive_or_steal_task( __TBB_atomic reference_count& completion_ref_count, bool return_if_no_work );
|
||||
|
||||
}; // class custom_scheduler<>
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// custom_scheduler methods
|
||||
//------------------------------------------------------------------------
|
||||
template<typename SchedulerTraits>
|
||||
task* custom_scheduler<SchedulerTraits>::receive_or_steal_task( __TBB_atomic reference_count& completion_ref_count,
|
||||
bool return_if_no_work ) {
|
||||
task* t = NULL;
|
||||
bool outermost_dispatch_level = return_if_no_work || master_outermost_level();
|
||||
bool can_steal_here = can_steal();
|
||||
my_inbox.set_is_idle( true );
|
||||
#if __TBB_HOARD_NONLOCAL_TASKS
|
||||
__TBB_ASSERT(!my_nonlocal_free_list, NULL);
|
||||
#endif
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( return_if_no_work && my_arena->my_skipped_fifo_priority ) {
|
||||
// This thread can dequeue FIFO tasks, and some priority levels of
|
||||
// FIFO tasks have been bypassed (to prevent deadlock caused by
|
||||
// dynamic priority changes in nested task group hierarchy).
|
||||
intptr_t skipped_priority = my_arena->my_skipped_fifo_priority;
|
||||
if ( my_arena->my_skipped_fifo_priority.compare_and_swap(0, skipped_priority) == skipped_priority &&
|
||||
skipped_priority > my_arena->my_top_priority )
|
||||
{
|
||||
my_market->update_arena_priority( *my_arena, skipped_priority );
|
||||
}
|
||||
}
|
||||
task_stream *ts;
|
||||
#else /* !__TBB_TASK_PRIORITY */
|
||||
task_stream *ts = &my_arena->my_task_stream;
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
// TODO: Try to find a place to reset my_limit (under market's lock)
|
||||
// The number of slots potentially used in the arena. Updated once in a while, as my_limit changes rarely.
|
||||
size_t n = my_arena->my_limit-1;
|
||||
int yield_count = 0;
|
||||
// The state "failure_count==-1" is used only when itt_possible is true,
|
||||
// and denotes that a sync_prepare has not yet been issued.
|
||||
for( int failure_count = -static_cast<int>(SchedulerTraits::itt_possible);; ++failure_count) {
|
||||
__TBB_ASSERT( my_arena->my_limit > 0, NULL );
|
||||
__TBB_ASSERT( my_arena_index <= n, NULL );
|
||||
if( completion_ref_count==1 ) {
|
||||
if( SchedulerTraits::itt_possible ) {
|
||||
if( failure_count!=-1 ) {
|
||||
ITT_NOTIFY(sync_prepare, &completion_ref_count);
|
||||
// Notify Intel(R) Thread Profiler that thread has stopped spinning.
|
||||
ITT_NOTIFY(sync_acquired, this);
|
||||
}
|
||||
ITT_NOTIFY(sync_acquired, &completion_ref_count);
|
||||
}
|
||||
__TBB_ASSERT( !t, NULL );
|
||||
__TBB_control_consistency_helper(); // on ref_count
|
||||
break; // exit stealing loop and return;
|
||||
}
|
||||
// Check if the resource manager requires our arena to relinquish some threads
|
||||
if ( return_if_no_work && my_arena->my_num_workers_allotted < my_arena->num_workers_active() ) {
|
||||
#if !__TBB_TASK_ARENA
|
||||
__TBB_ASSERT( is_worker(), NULL );
|
||||
#endif
|
||||
if( SchedulerTraits::itt_possible && failure_count != -1 )
|
||||
ITT_NOTIFY(sync_cancel, this);
|
||||
return NULL;
|
||||
}
|
||||
#if __TBB_TASK_PRIORITY
|
||||
ts = &my_arena->my_task_stream[my_arena->my_top_priority];
|
||||
#endif
|
||||
// Check if there are tasks mailed to this thread via task-to-thread affinity mechanism.
|
||||
__TBB_ASSERT(my_affinity_id, NULL);
|
||||
if ( n && !my_inbox.empty() && (t = get_mailbox_task()) ) {
|
||||
GATHER_STATISTIC( ++my_counters.mails_received );
|
||||
}
|
||||
// Check if there are tasks in starvation-resistant stream.
|
||||
// Only allowed for workers with empty stack, which is identified by return_if_no_work.
|
||||
else if ( outermost_dispatch_level && !ts->empty() && (t = ts->pop( my_arena_slot->hint_for_pop)) ) {
|
||||
ITT_NOTIFY(sync_acquired, ts);
|
||||
// just proceed with the obtained task
|
||||
}
|
||||
#if __TBB_TASK_PRIORITY
|
||||
// Check if any earlier offloaded non-top priority tasks become returned to the top level
|
||||
else if ( my_offloaded_tasks && (t=reload_tasks()) ) {
|
||||
// just proceed with the obtained task
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
else if ( can_steal_here && n ) {
|
||||
// Try to steal a task from a random victim.
|
||||
size_t k = my_random.get() % n;
|
||||
arena_slot* victim = &my_arena->my_slots[k];
|
||||
// The following condition excludes the master that might have
|
||||
// already taken our previous place in the arena from the list .
|
||||
// of potential victims. But since such a situation can take
|
||||
// place only in case of significant oversubscription, keeping
|
||||
// the checks simple seems to be preferable to complicating the code.
|
||||
if( k >= my_arena_index )
|
||||
++victim; // Adjusts random distribution to exclude self
|
||||
task **pool = victim->task_pool;
|
||||
if( pool == EmptyTaskPool || !(t = steal_task( *victim )) )
|
||||
goto fail;
|
||||
if( is_proxy(*t) ) {
|
||||
task_proxy &tp = *(task_proxy*)t;
|
||||
t = tp.extract_task<task_proxy::pool_bit>();
|
||||
if ( !t ) {
|
||||
// Proxy was empty, so it's our responsibility to free it
|
||||
free_task<no_cache_small_task>(tp);
|
||||
goto fail;
|
||||
}
|
||||
GATHER_STATISTIC( ++my_counters.proxies_stolen );
|
||||
}
|
||||
t->prefix().extra_state |= es_task_is_stolen;
|
||||
if( is_version_3_task(*t) ) {
|
||||
my_innermost_running_task = t;
|
||||
t->prefix().owner = this;
|
||||
t->note_affinity( my_affinity_id );
|
||||
}
|
||||
GATHER_STATISTIC( ++my_counters.steals_committed );
|
||||
} // end of stealing branch
|
||||
else
|
||||
goto fail;
|
||||
// A task was successfully obtained somewhere
|
||||
__TBB_ASSERT(t,NULL);
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
my_arena->my_observers.notify_entry_observers( my_last_local_observer, is_worker() );
|
||||
the_global_observer_list.notify_entry_observers( my_last_global_observer, is_worker() );
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
if ( SchedulerTraits::itt_possible && failure_count != -1 ) {
|
||||
// FIXME - might be victim, or might be selected from a mailbox
|
||||
// Notify Intel(R) Thread Profiler that thread has stopped spinning.
|
||||
ITT_NOTIFY(sync_acquired, this);
|
||||
}
|
||||
break; // exit stealing loop and return
|
||||
fail:
|
||||
GATHER_STATISTIC( ++my_counters.steals_failed );
|
||||
if( SchedulerTraits::itt_possible && failure_count==-1 ) {
|
||||
// The first attempt to steal work failed, so notify Intel(R) Thread Profiler that
|
||||
// the thread has started spinning. Ideally, we would do this notification
|
||||
// *before* the first failed attempt to steal, but at that point we do not
|
||||
// know that the steal will fail.
|
||||
ITT_NOTIFY(sync_prepare, this);
|
||||
failure_count = 0;
|
||||
}
|
||||
// Pause, even if we are going to yield, because the yield might return immediately.
|
||||
__TBB_Pause(PauseTime);
|
||||
const int failure_threshold = 2*int(n+1);
|
||||
if( failure_count>=failure_threshold ) {
|
||||
#if __TBB_YIELD2P
|
||||
failure_count = 0;
|
||||
#else
|
||||
failure_count = failure_threshold;
|
||||
#endif
|
||||
__TBB_Yield();
|
||||
#if __TBB_TASK_PRIORITY
|
||||
// Check if there are tasks abandoned by other workers
|
||||
if ( my_arena->my_orphaned_tasks ) {
|
||||
// Epoch must be advanced before seizing the list pointer
|
||||
++my_arena->my_abandonment_epoch;
|
||||
task* orphans = (task*)__TBB_FetchAndStoreW( &my_arena->my_orphaned_tasks, 0 );
|
||||
if ( orphans ) {
|
||||
task** link = NULL;
|
||||
// Get local counter out of the way (we've just brought in external tasks)
|
||||
my_local_reload_epoch--;
|
||||
t = reload_tasks( orphans, link, effective_reference_priority() );
|
||||
if ( orphans ) {
|
||||
*link = my_offloaded_tasks;
|
||||
if ( !my_offloaded_tasks )
|
||||
my_offloaded_task_list_tail_link = link;
|
||||
my_offloaded_tasks = orphans;
|
||||
}
|
||||
__TBB_ASSERT( !my_offloaded_tasks == !my_offloaded_task_list_tail_link, NULL );
|
||||
if ( t ) {
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_cancel, this);
|
||||
break; // exit stealing loop and return
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
const int yield_threshold = 100;
|
||||
if( yield_count++ >= yield_threshold ) {
|
||||
// When a worker thread has nothing to do, return it to RML.
|
||||
// For purposes of affinity support, the thread is considered idle while in RML.
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if( return_if_no_work || my_arena->my_top_priority > my_arena->my_bottom_priority ) {
|
||||
if ( my_arena->is_out_of_work() && return_if_no_work ) {
|
||||
#else /* !__TBB_TASK_PRIORITY */
|
||||
if ( return_if_no_work && my_arena->is_out_of_work() ) {
|
||||
#endif /* !__TBB_TASK_PRIORITY */
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_cancel, this);
|
||||
return NULL;
|
||||
}
|
||||
#if __TBB_TASK_PRIORITY
|
||||
}
|
||||
if ( my_offloaded_tasks ) {
|
||||
// Safeguard against any sloppiness in managing reload epoch
|
||||
// counter (e.g. on the hot path because of performance reasons).
|
||||
my_local_reload_epoch--;
|
||||
// Break the deadlock caused by a higher priority dispatch loop
|
||||
// stealing and offloading a lower priority task. Priority check
|
||||
// at the stealing moment cannot completely preclude such cases
|
||||
// because priorities can changes dynamically.
|
||||
if ( !return_if_no_work && *my_ref_top_priority > my_arena->my_top_priority ) {
|
||||
GATHER_STATISTIC( ++my_counters.prio_ref_fixups );
|
||||
my_ref_top_priority = &my_arena->my_top_priority;
|
||||
// it's expected that only outermost workers can use global reload epoch
|
||||
__TBB_ASSERT(!worker_outermost_level(), NULL);
|
||||
__TBB_ASSERT(my_ref_reload_epoch == &my_arena->my_reload_epoch, NULL);
|
||||
}
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
} // end of arena snapshot branch
|
||||
// If several attempts did not find work, re-read the arena limit.
|
||||
n = my_arena->my_limit-1;
|
||||
} // end of yielding branch
|
||||
} // end of nonlocal task retrieval loop
|
||||
my_inbox.set_is_idle( false );
|
||||
return t;
|
||||
}
|
||||
|
||||
template<typename SchedulerTraits>
|
||||
void custom_scheduler<SchedulerTraits>::local_wait_for_all( task& parent, task* child ) {
|
||||
__TBB_ASSERT( governor::is_set(this), NULL );
|
||||
__TBB_ASSERT( parent.ref_count() >= (child && child->parent() == &parent ? 2 : 1), "ref_count is too small" );
|
||||
assert_task_pool_valid();
|
||||
// Using parent's refcount in sync_prepare (in the stealing loop below) is
|
||||
// a workaround for TP. We need to name it here to display correctly in Ampl.
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_SYNC_CREATE(&parent.prefix().ref_count, SyncType_Scheduler, SyncObj_TaskStealingLoop);
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_ASSERT( parent.prefix().context || (is_worker() && &parent == my_dummy_task), "parent task does not have context" );
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
task* t = child;
|
||||
// Constant all_local_work_done is an unreachable refcount value that prevents
|
||||
// early quitting the dispatch loop. It is defined to be in the middle of the range
|
||||
// of negative values representable by the reference_count type.
|
||||
static const reference_count
|
||||
// For normal dispatch loops
|
||||
parents_work_done = 1,
|
||||
// For termination dispatch loops in masters
|
||||
all_local_work_done = (reference_count)3 << (sizeof(reference_count) * 8 - 2);
|
||||
reference_count quit_point;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_ASSERT( (uintptr_t)*my_ref_top_priority < (uintptr_t)num_priority_levels, NULL );
|
||||
volatile intptr_t *old_ref_top_priority = my_ref_top_priority;
|
||||
// When entering nested parallelism level market level counter
|
||||
// must be replaced with the one local to this arena.
|
||||
volatile uintptr_t *old_ref_reload_epoch = my_ref_reload_epoch;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
task* old_dispatching_task = my_dispatching_task;
|
||||
my_dispatching_task = my_innermost_running_task;
|
||||
if( master_outermost_level() ) {
|
||||
// We are in the outermost task dispatch loop of a master thread or a worker which mimics master
|
||||
__TBB_ASSERT( !is_worker() || my_dispatching_task != old_dispatching_task, NULL );
|
||||
quit_point = &parent == my_dummy_task ? all_local_work_done : parents_work_done;
|
||||
} else {
|
||||
quit_point = parents_work_done;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
if ( &parent != my_dummy_task ) {
|
||||
// We are in a nested dispatch loop.
|
||||
// Market or arena priority must not prevent child tasks from being
|
||||
// executed so that dynamic priority changes did not cause deadlock.
|
||||
my_ref_top_priority = &parent.prefix().context->my_priority;
|
||||
my_ref_reload_epoch = &my_arena->my_reload_epoch;
|
||||
if(my_ref_reload_epoch != old_ref_reload_epoch)
|
||||
my_local_reload_epoch = *my_ref_reload_epoch-1;
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
}
|
||||
|
||||
cpu_ctl_env_helper cpu_ctl_helper;
|
||||
if ( t )
|
||||
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
|
||||
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
// Infinite safeguard EH loop
|
||||
for (;;) {
|
||||
try {
|
||||
#endif /* TBB_USE_EXCEPTIONS */
|
||||
// Outer loop receives tasks from global environment (via mailbox, FIFO queue(s),
|
||||
// and by stealing from other threads' task pools).
|
||||
// All exit points from the dispatch loop are located in its immediate scope.
|
||||
for(;;) {
|
||||
// Middle loop retrieves tasks from the local task pool.
|
||||
for(;;) {
|
||||
// Inner loop evaluates tasks coming from nesting loops and those returned
|
||||
// by just executed tasks (bypassing spawn or enqueue calls).
|
||||
while(t) {
|
||||
__TBB_ASSERT( my_inbox.is_idle_state(false), NULL );
|
||||
__TBB_ASSERT(!is_proxy(*t),"unexpected proxy");
|
||||
__TBB_ASSERT( t->prefix().owner, NULL );
|
||||
assert_task_valid(*t);
|
||||
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_ASSERT
|
||||
assert_context_valid(t->prefix().context);
|
||||
if ( !t->prefix().context->my_cancellation_requested )
|
||||
#endif
|
||||
__TBB_ASSERT( 1L<<t->state() & (1L<<task::allocated|1L<<task::ready|1L<<task::reexecute), NULL );
|
||||
assert_task_pool_valid();
|
||||
#if __TBB_TASK_PRIORITY
|
||||
intptr_t p = priority(*t);
|
||||
if ( p != *my_ref_top_priority && (t->prefix().extra_state & es_task_enqueued) == 0) {
|
||||
assert_priority_valid(p);
|
||||
if ( p != my_arena->my_top_priority ) {
|
||||
my_market->update_arena_priority( *my_arena, p );
|
||||
}
|
||||
if ( p < effective_reference_priority() ) {
|
||||
if ( !my_offloaded_tasks ) {
|
||||
my_offloaded_task_list_tail_link = &t->prefix().next_offloaded;
|
||||
// Erase possible reference to the owner scheduler (next_offloaded is a union member)
|
||||
*my_offloaded_task_list_tail_link = NULL;
|
||||
}
|
||||
offload_task( *t, p );
|
||||
if ( in_arena() ) {
|
||||
t = winnow_task_pool();
|
||||
if ( t )
|
||||
continue;
|
||||
}
|
||||
else {
|
||||
// Mark arena as full to unlock arena priority level adjustment
|
||||
// by arena::is_out_of_work(), and ensure worker's presence.
|
||||
my_arena->advertise_new_work<false>();
|
||||
}
|
||||
goto stealing_ground;
|
||||
}
|
||||
}
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
task* t_next = NULL;
|
||||
my_innermost_running_task = t;
|
||||
t->prefix().owner = this;
|
||||
t->prefix().state = task::executing;
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
if ( !t->prefix().context->my_cancellation_requested )
|
||||
#endif
|
||||
{
|
||||
GATHER_STATISTIC( ++my_counters.tasks_executed );
|
||||
GATHER_STATISTIC( my_counters.avg_arena_concurrency += my_arena->num_workers_active() );
|
||||
GATHER_STATISTIC( my_counters.avg_assigned_workers += my_arena->my_num_workers_allotted );
|
||||
#if __TBB_TASK_PRIORITY
|
||||
GATHER_STATISTIC( my_counters.avg_arena_prio += p );
|
||||
GATHER_STATISTIC( my_counters.avg_market_prio += my_market->my_global_top_priority );
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
ITT_STACK(SchedulerTraits::itt_possible, callee_enter, t->prefix().context->itt_caller);
|
||||
t_next = t->execute();
|
||||
ITT_STACK(SchedulerTraits::itt_possible, callee_leave, t->prefix().context->itt_caller);
|
||||
if (t_next) {
|
||||
__TBB_ASSERT( t_next->state()==task::allocated,
|
||||
"if task::execute() returns task, it must be marked as allocated" );
|
||||
reset_extra_state(t_next);
|
||||
#if TBB_USE_ASSERT
|
||||
affinity_id next_affinity=t_next->prefix().affinity;
|
||||
if (next_affinity != 0 && next_affinity != my_affinity_id)
|
||||
GATHER_STATISTIC( ++my_counters.affinity_ignored );
|
||||
#endif
|
||||
}
|
||||
}
|
||||
assert_task_pool_valid();
|
||||
switch( t->state() ) {
|
||||
case task::executing: {
|
||||
task* s = t->parent();
|
||||
__TBB_ASSERT( my_innermost_running_task==t, NULL );
|
||||
__TBB_ASSERT( t->prefix().ref_count==0, "Task still has children after it has been executed" );
|
||||
t->~task();
|
||||
if( s )
|
||||
tally_completion_of_predecessor(*s, t_next);
|
||||
free_task<no_hint>( *t );
|
||||
assert_task_pool_valid();
|
||||
break;
|
||||
}
|
||||
|
||||
case task::recycle: // set by recycle_as_safe_continuation()
|
||||
t->prefix().state = task::allocated;
|
||||
#if __TBB_RECYCLE_TO_ENQUEUE
|
||||
case task::to_enqueue: // set by recycle_to_enqueue()
|
||||
#endif
|
||||
__TBB_ASSERT( t_next != t, "a task returned from method execute() can not be recycled in another way" );
|
||||
reset_extra_state(t);
|
||||
// for safe continuation, need atomically decrement ref_count;
|
||||
tally_completion_of_predecessor(*t, t_next);
|
||||
assert_task_pool_valid();
|
||||
break;
|
||||
|
||||
case task::reexecute: // set by recycle_to_reexecute()
|
||||
__TBB_ASSERT( t_next, "reexecution requires that method execute() return another task" );
|
||||
__TBB_ASSERT( t_next != t, "a task returned from method execute() can not be recycled in another way" );
|
||||
t->prefix().state = task::allocated;
|
||||
reset_extra_state(t);
|
||||
local_spawn( *t, t->prefix().next );
|
||||
assert_task_pool_valid();
|
||||
break;
|
||||
case task::allocated:
|
||||
reset_extra_state(t);
|
||||
break;
|
||||
#if TBB_USE_ASSERT
|
||||
case task::ready:
|
||||
__TBB_ASSERT( false, "task is in READY state upon return from method execute()" );
|
||||
break;
|
||||
default:
|
||||
__TBB_ASSERT( false, "illegal state" );
|
||||
#else
|
||||
default: // just to shut up some compilation warnings
|
||||
break;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
}
|
||||
GATHER_STATISTIC( t_next ? ++my_counters.spawns_bypassed : 0 );
|
||||
t = t_next;
|
||||
} // end of scheduler bypass loop
|
||||
|
||||
assert_task_pool_valid();
|
||||
if ( parent.prefix().ref_count == quit_point ) {
|
||||
__TBB_ASSERT( quit_point != all_local_work_done, NULL );
|
||||
__TBB_control_consistency_helper(); // on ref_count
|
||||
ITT_NOTIFY(sync_acquired, &parent.prefix().ref_count);
|
||||
goto done;
|
||||
}
|
||||
if ( in_arena() ) {
|
||||
t = get_task();
|
||||
}
|
||||
else {
|
||||
__TBB_ASSERT( is_quiescent_local_task_pool_reset(), NULL );
|
||||
break;
|
||||
}
|
||||
__TBB_ASSERT(!t || !is_proxy(*t),"unexpected proxy");
|
||||
assert_task_pool_valid();
|
||||
|
||||
if ( !t ) break;
|
||||
|
||||
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
|
||||
}; // end of local task pool retrieval loop
|
||||
|
||||
#if __TBB_TASK_PRIORITY
|
||||
stealing_ground:
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
#if __TBB_HOARD_NONLOCAL_TASKS
|
||||
// before stealing, previously stolen task objects are returned
|
||||
for (; my_nonlocal_free_list; my_nonlocal_free_list = t ) {
|
||||
t = my_nonlocal_free_list->prefix().next;
|
||||
free_nonlocal_small_task( *my_nonlocal_free_list );
|
||||
}
|
||||
#endif
|
||||
if ( quit_point == all_local_work_done ) {
|
||||
__TBB_ASSERT( !in_arena() && is_quiescent_local_task_pool_reset(), NULL );
|
||||
__TBB_ASSERT( !worker_outermost_level(), NULL );
|
||||
my_innermost_running_task = my_dispatching_task;
|
||||
my_dispatching_task = old_dispatching_task;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
my_ref_top_priority = old_ref_top_priority;
|
||||
if(my_ref_reload_epoch != old_ref_reload_epoch)
|
||||
my_local_reload_epoch = *old_ref_reload_epoch-1;
|
||||
my_ref_reload_epoch = old_ref_reload_epoch;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
return;
|
||||
}
|
||||
// The following assertion may be falsely triggered in the presence of enqueued tasks
|
||||
//__TBB_ASSERT( my_arena->my_max_num_workers > 0 || my_market->my_ref_count > 1
|
||||
// || parent.prefix().ref_count == 1, "deadlock detected" );
|
||||
|
||||
// Dispatching task pointer is NULL *iff* this is a worker thread in its outermost
|
||||
// dispatch loop (i.e. its execution stack is empty). In this case it should exit it
|
||||
// either when there is no more work in the current arena, or when revoked by the market.
|
||||
|
||||
t = receive_or_steal_task( parent.prefix().ref_count, worker_outermost_level() );
|
||||
if ( !t )
|
||||
goto done;
|
||||
__TBB_ASSERT(!is_proxy(*t),"unexpected proxy");
|
||||
|
||||
// The user can capture another the FPU settings to the context so the
|
||||
// cached data in the helper can be out-of-date and we cannot do fast
|
||||
// check.
|
||||
cpu_ctl_helper.set_env( __TBB_CONTEXT_ARG1(t->prefix().context) );
|
||||
} // end of infinite stealing loop
|
||||
#if TBB_USE_EXCEPTIONS
|
||||
__TBB_ASSERT( false, "Must never get here" );
|
||||
} // end of try-block
|
||||
TbbCatchAll( t->prefix().context );
|
||||
// Complete post-processing ...
|
||||
if( t->state() == task::recycle
|
||||
#if __TBB_RECYCLE_TO_ENQUEUE
|
||||
// TODO: the enqueue semantics gets lost below, consider reimplementing
|
||||
|| t->state() == task::to_enqueue
|
||||
#endif
|
||||
) {
|
||||
// ... for recycled tasks to atomically decrement ref_count
|
||||
t->prefix().state = task::allocated;
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_releasing, &t->prefix().ref_count);
|
||||
if( __TBB_FetchAndDecrementWrelease(&t->prefix().ref_count)==1 ) {
|
||||
if( SchedulerTraits::itt_possible )
|
||||
ITT_NOTIFY(sync_acquired, &t->prefix().ref_count);
|
||||
}else{
|
||||
t = NULL;
|
||||
}
|
||||
}
|
||||
} // end of infinite EH loop
|
||||
__TBB_ASSERT( false, "Must never get here too" );
|
||||
#endif /* TBB_USE_EXCEPTIONS */
|
||||
done:
|
||||
my_innermost_running_task = my_dispatching_task;
|
||||
my_dispatching_task = old_dispatching_task;
|
||||
#if __TBB_TASK_PRIORITY
|
||||
my_ref_top_priority = old_ref_top_priority;
|
||||
if(my_ref_reload_epoch != old_ref_reload_epoch)
|
||||
my_local_reload_epoch = *old_ref_reload_epoch-1;
|
||||
my_ref_reload_epoch = old_ref_reload_epoch;
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
if ( !ConcurrentWaitsEnabled(parent) ) {
|
||||
if ( parent.prefix().ref_count != parents_work_done ) {
|
||||
// This is a worker that was revoked by the market.
|
||||
#if __TBB_TASK_ARENA
|
||||
__TBB_ASSERT( worker_outermost_level(),
|
||||
"Worker thread exits nested dispatch loop prematurely" );
|
||||
#else
|
||||
__TBB_ASSERT( is_worker() && worker_outermost_level(),
|
||||
"Worker thread exits nested dispatch loop prematurely" );
|
||||
#endif
|
||||
return;
|
||||
}
|
||||
parent.prefix().ref_count = 0;
|
||||
}
|
||||
#if TBB_USE_ASSERT
|
||||
parent.prefix().extra_state &= ~es_ref_count_active;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_ASSERT(parent.prefix().context && default_context(), NULL);
|
||||
task_group_context* parent_ctx = parent.prefix().context;
|
||||
if ( parent_ctx->my_cancellation_requested ) {
|
||||
task_group_context::exception_container_type *pe = parent_ctx->my_exception;
|
||||
if ( master_outermost_level() && parent_ctx == default_context() ) {
|
||||
// We are in the outermost task dispatch loop of a master thread, and
|
||||
// the whole task tree has been collapsed. So we may clear cancellation data.
|
||||
parent_ctx->my_cancellation_requested = 0;
|
||||
// TODO: Add assertion that master's dummy task context does not have children
|
||||
parent_ctx->my_state &= ~(uintptr_t)task_group_context::may_have_children;
|
||||
}
|
||||
if ( pe ) {
|
||||
// On Windows, FPU control settings changed in the helper destructor are not visible
|
||||
// outside a catch block. So restore the default settings manually before rethrowing
|
||||
// the exception.
|
||||
cpu_ctl_helper.restore_default();
|
||||
pe->throw_self();
|
||||
}
|
||||
}
|
||||
__TBB_ASSERT(!is_worker() || !CancellationInfoPresent(*my_dummy_task),
|
||||
"Worker's dummy task context modified");
|
||||
__TBB_ASSERT(!master_outermost_level() || !CancellationInfoPresent(*my_dummy_task),
|
||||
"Unexpected exception or cancellation data in the master's dummy task");
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
assert_task_pool_valid();
|
||||
}
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* _TBB_custom_scheduler_H */
|
||||
@@ -0,0 +1,560 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "dynamic_link.h"
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/*
|
||||
This file is used by both TBB and OpenMP RTL. Do not use __TBB_ASSERT() macro
|
||||
and runtime_warning() function because they are not available in OpenMP. Use
|
||||
LIBRARY_ASSERT and DYNAMIC_LINK_WARNING instead.
|
||||
*/
|
||||
|
||||
#include <cstdarg> // va_list etc.
|
||||
#if _WIN32
|
||||
#include <malloc.h>
|
||||
|
||||
// Unify system calls
|
||||
#define dlopen( name, flags ) LoadLibraryA( name )
|
||||
#define dlsym( handle, name ) GetProcAddress( handle, name )
|
||||
#define dlclose( handle ) ( ! FreeLibrary( handle ) )
|
||||
#define dlerror() GetLastError()
|
||||
#ifndef PATH_MAX
|
||||
#define PATH_MAX MAX_PATH
|
||||
#endif
|
||||
#else /* _WIN32 */
|
||||
#include <dlfcn.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
#endif /* _WIN32 */
|
||||
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
//TODO: use function attribute for weak symbols instead of the pragma.
|
||||
#pragma weak dlopen
|
||||
#pragma weak dlsym
|
||||
#pragma weak dlclose
|
||||
#pragma weak dlerror
|
||||
#pragma weak dladdr
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
|
||||
#include "tbb/tbb_misc.h"
|
||||
|
||||
#define __USE_TBB_ATOMICS ( !(__linux__&&__ia64__) || __TBB_BUILD )
|
||||
#define __USE_STATIC_DL_INIT (!__ANDROID__)
|
||||
|
||||
#if !__USE_TBB_ATOMICS
|
||||
#include <pthread.h>
|
||||
#endif
|
||||
|
||||
/*
|
||||
dynamic_link is a common interface for searching for required symbols in an
|
||||
executable and dynamic libraries.
|
||||
|
||||
dynamic_link provides certain guarantees:
|
||||
1. Either all or none of the requested symbols are resolved. Moreover, if
|
||||
symbols are not resolved, the dynamic_link_descriptor table is not modified;
|
||||
2. All returned symbols have secured life time: this means that none of them
|
||||
can be invalidated until dynamic_unlink is called;
|
||||
3. Any loaded library is loaded only via the full path. The full path is that
|
||||
from which the runtime itself was loaded. (This is done to avoid security
|
||||
issues caused by loading libraries from insecure paths).
|
||||
|
||||
dynamic_link searches for the requested symbols in three stages, stopping as
|
||||
soon as all of the symbols have been resolved.
|
||||
|
||||
1. Search the global scope:
|
||||
a. On Windows: dynamic_link tries to obtain the handle of the requested
|
||||
library and if it succeeds it resolves the symbols via that handle.
|
||||
b. On Linux: dynamic_link tries to search for the symbols in the global
|
||||
scope via the main program handle. If the symbols are present in the global
|
||||
scope their life time is not guaranteed (since dynamic_link does not know
|
||||
anything about the library from which they are exported). Therefore it
|
||||
tries to "pin" the symbols by obtaining the library name and reopening it.
|
||||
dlopen may fail to reopen the library in two cases:
|
||||
i. The symbols are exported from the executable. Currently dynamic _link
|
||||
cannot handle this situation, so it will not find these symbols in this
|
||||
step.
|
||||
ii. The necessary library has been unloaded and cannot be reloaded. It
|
||||
seems there is nothing that can be done in this case. No symbols are
|
||||
returned.
|
||||
|
||||
2. Dynamic load: an attempt is made to load the requested library via the
|
||||
full path.
|
||||
The full path used is that from which the runtime itself was loaded. If the
|
||||
library can be loaded, then an attempt is made to resolve the requested
|
||||
symbols in the newly loaded library.
|
||||
If the symbols are not found the library is unloaded.
|
||||
|
||||
3. Weak symbols: if weak symbols are available they are returned.
|
||||
*/
|
||||
|
||||
OPEN_INTERNAL_NAMESPACE
|
||||
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED
|
||||
|
||||
#if !defined(DYNAMIC_LINK_WARNING) && !__TBB_WIN8UI_SUPPORT
|
||||
// Report runtime errors and continue.
|
||||
#define DYNAMIC_LINK_WARNING dynamic_link_warning
|
||||
static void dynamic_link_warning( dynamic_link_error_t code, ... ) {
|
||||
(void) code;
|
||||
} // library_warning
|
||||
#endif /* DYNAMIC_LINK_WARNING */
|
||||
static bool resolve_symbols( dynamic_link_handle module, const dynamic_link_descriptor descriptors[], size_t required )
|
||||
{
|
||||
LIBRARY_ASSERT( module != NULL, "Module handle is NULL" );
|
||||
if ( module == NULL )
|
||||
return false;
|
||||
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
if ( !dlsym ) return false;
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
|
||||
const size_t n_desc=20; // Usually we don't have more than 20 descriptors per library
|
||||
LIBRARY_ASSERT( required <= n_desc, "Too many descriptors is required" );
|
||||
if ( required > n_desc ) return false;
|
||||
pointer_to_handler h[n_desc];
|
||||
|
||||
for ( size_t k = 0; k < required; ++k ) {
|
||||
dynamic_link_descriptor const & desc = descriptors[k];
|
||||
pointer_to_handler addr = (pointer_to_handler)dlsym( module, desc.name );
|
||||
if ( !addr ) {
|
||||
return false;
|
||||
}
|
||||
h[k] = addr;
|
||||
}
|
||||
|
||||
// Commit the entry points.
|
||||
// Cannot use memset here, because the writes must be atomic.
|
||||
for( size_t k = 0; k < required; ++k )
|
||||
*descriptors[k].handler = h[k];
|
||||
return true;
|
||||
}
|
||||
|
||||
#if __TBB_WIN8UI_SUPPORT
|
||||
bool dynamic_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required, dynamic_link_handle*, int flags ) {
|
||||
dynamic_link_handle tmp_handle = NULL;
|
||||
TCHAR wlibrary[256];
|
||||
if ( MultiByteToWideChar(CP_UTF8, 0, library, -1, wlibrary, 255) == 0 ) return false;
|
||||
if ( flags & DYNAMIC_LINK_LOAD )
|
||||
tmp_handle = LoadPackagedLibrary( wlibrary, 0 );
|
||||
if (tmp_handle != NULL){
|
||||
return resolve_symbols(tmp_handle, descriptors, required);
|
||||
}else{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
void dynamic_unlink( dynamic_link_handle ) {
|
||||
}
|
||||
void dynamic_unlink_all() {
|
||||
}
|
||||
#else
|
||||
/*
|
||||
There is a security issue on Windows: LoadLibrary() may load and execute malicious code.
|
||||
See http://www.microsoft.com/technet/security/advisory/2269637.mspx for details.
|
||||
To avoid the issue, we have to pass full path (not just library name) to LoadLibrary. This
|
||||
function constructs full path to the specified library (it is assumed the library located
|
||||
side-by-side with the tbb.dll.
|
||||
|
||||
The function constructs absolute path for given relative path. Important: Base directory is not
|
||||
current one, it is the directory tbb.dll loaded from.
|
||||
|
||||
Example:
|
||||
Let us assume "tbb.dll" is located in "c:\program files\common\intel\" directory, e. g.
|
||||
absolute path of tbb library is "c:\program files\common\intel\tbb.dll". Absolute path for
|
||||
"tbbmalloc.dll" would be "c:\program files\common\intel\tbbmalloc.dll". Absolute path for
|
||||
"malloc\tbbmalloc.dll" would be "c:\program files\common\intel\malloc\tbbmalloc.dll".
|
||||
*/
|
||||
|
||||
// Struct handle_storage is used by dynamic_link routine to store handles of
|
||||
// all loaded or pinned dynamic libraries. When TBB is shut down, it calls
|
||||
// dynamic_unlink_all() that unloads modules referenced by handle_storage.
|
||||
// This struct should not have any constructors since it may be used before
|
||||
// the constructor is called.
|
||||
#define MAX_LOADED_MODULES 8 // The number of maximum possible modules which can be loaded
|
||||
|
||||
struct handle_storage {
|
||||
#if __USE_TBB_ATOMICS
|
||||
::tbb::atomic<size_t> my_size;
|
||||
#else
|
||||
size_t my_size;
|
||||
pthread_spinlock_t my_lock;
|
||||
#endif
|
||||
dynamic_link_handle my_handles[MAX_LOADED_MODULES];
|
||||
|
||||
void add_handle(const dynamic_link_handle &handle) {
|
||||
#if !__USE_TBB_ATOMICS
|
||||
int res = pthread_spin_lock( &my_lock );
|
||||
LIBRARY_ASSERT( res==0, "pthread_spin_lock failed" );
|
||||
#endif
|
||||
const size_t ind = my_size++;
|
||||
#if !__USE_TBB_ATOMICS
|
||||
res = pthread_spin_unlock( &my_lock );
|
||||
LIBRARY_ASSERT( res==0, "pthread_spin_unlock failed" );
|
||||
#endif
|
||||
LIBRARY_ASSERT( ind < MAX_LOADED_MODULES, "Too many modules are loaded" );
|
||||
my_handles[ind] = handle;
|
||||
}
|
||||
|
||||
void free_handles() {
|
||||
const size_t size = my_size;
|
||||
for (size_t i=0; i<size; ++i)
|
||||
dynamic_unlink( my_handles[i] );
|
||||
}
|
||||
};
|
||||
|
||||
handle_storage handles;
|
||||
|
||||
#if __USE_TBB_ATOMICS
|
||||
static void atomic_once ( void (*func) (void), tbb::atomic< tbb::internal::do_once_state > &once_state ) {
|
||||
tbb::internal::atomic_do_once( func, once_state );
|
||||
}
|
||||
#define ATOMIC_ONCE_DECL( var ) tbb::atomic< tbb::internal::do_once_state > var
|
||||
#else
|
||||
static void atomic_once ( void (*func) (), pthread_once_t &once_state ) {
|
||||
pthread_once( &once_state, func );
|
||||
}
|
||||
#define ATOMIC_ONCE_DECL( var ) pthread_once_t var = PTHREAD_ONCE_INIT
|
||||
#endif
|
||||
|
||||
ATOMIC_ONCE_DECL( init_dl_data_state );
|
||||
|
||||
static struct _ap_data {
|
||||
char _path[PATH_MAX+1];
|
||||
size_t _len;
|
||||
} ap_data;
|
||||
|
||||
static void init_ap_data() {
|
||||
#if _WIN32
|
||||
// Get handle of our DLL first.
|
||||
HMODULE handle;
|
||||
BOOL brc = GetModuleHandleExA(
|
||||
GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
|
||||
(LPSTR)( & dynamic_link ), // any function inside the library can be used for the address
|
||||
& handle
|
||||
);
|
||||
if ( !brc ) { // Error occurred.
|
||||
int err = GetLastError();
|
||||
DYNAMIC_LINK_WARNING( dl_sys_fail, "GetModuleHandleEx", err );
|
||||
return;
|
||||
}
|
||||
// Now get path to our DLL.
|
||||
DWORD drc = GetModuleFileNameA( handle, ap_data._path, static_cast< DWORD >( PATH_MAX ) );
|
||||
if ( drc == 0 ) { // Error occurred.
|
||||
int err = GetLastError();
|
||||
DYNAMIC_LINK_WARNING( dl_sys_fail, "GetModuleFileName", err );
|
||||
return;
|
||||
}
|
||||
if ( drc >= PATH_MAX ) { // Buffer too short.
|
||||
DYNAMIC_LINK_WARNING( dl_buff_too_small );
|
||||
return;
|
||||
}
|
||||
// Find the position of the last backslash.
|
||||
char *backslash = strrchr( ap_data._path, '\\' );
|
||||
|
||||
if ( !backslash ) { // Backslash not found.
|
||||
LIBRARY_ASSERT( backslash!=NULL, "Unbelievable.");
|
||||
return;
|
||||
}
|
||||
LIBRARY_ASSERT( backslash >= ap_data._path, "Unbelievable.");
|
||||
ap_data._len = (size_t)(backslash - ap_data._path) + 1;
|
||||
*(backslash+1) = 0;
|
||||
#else
|
||||
// Get the library path
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
if ( !dladdr || !dlerror ) return;
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
Dl_info dlinfo;
|
||||
int res = dladdr( (void*)&dynamic_link, &dlinfo ); // any function inside the library can be used for the address
|
||||
if ( !res ) {
|
||||
char const * err = dlerror();
|
||||
DYNAMIC_LINK_WARNING( dl_sys_fail, "dladdr", err );
|
||||
return;
|
||||
} else {
|
||||
LIBRARY_ASSERT( dlinfo.dli_fname!=NULL, "Unbelievable." );
|
||||
}
|
||||
|
||||
char const *slash = strrchr( dlinfo.dli_fname, '/' );
|
||||
size_t fname_len=0;
|
||||
if ( slash ) {
|
||||
LIBRARY_ASSERT( slash >= dlinfo.dli_fname, "Unbelievable.");
|
||||
fname_len = (size_t)(slash - dlinfo.dli_fname) + 1;
|
||||
}
|
||||
|
||||
size_t rc;
|
||||
if ( dlinfo.dli_fname[0]=='/' ) {
|
||||
// The library path is absolute
|
||||
rc = 0;
|
||||
ap_data._len = 0;
|
||||
} else {
|
||||
// The library path is relative so get the current working directory
|
||||
if ( !getcwd( ap_data._path, sizeof(ap_data._path)/sizeof(ap_data._path[0]) ) ) {
|
||||
DYNAMIC_LINK_WARNING( dl_buff_too_small );
|
||||
return;
|
||||
}
|
||||
ap_data._len = strlen( ap_data._path );
|
||||
ap_data._path[ap_data._len++]='/';
|
||||
rc = ap_data._len;
|
||||
}
|
||||
|
||||
if ( fname_len>0 ) {
|
||||
if ( ap_data._len>PATH_MAX ) {
|
||||
DYNAMIC_LINK_WARNING( dl_buff_too_small );
|
||||
ap_data._len=0;
|
||||
return;
|
||||
}
|
||||
strncpy( ap_data._path+rc, dlinfo.dli_fname, fname_len );
|
||||
ap_data._len += fname_len;
|
||||
ap_data._path[ap_data._len]=0;
|
||||
}
|
||||
#endif /* _WIN32 */
|
||||
}
|
||||
|
||||
static void init_dl_data() {
|
||||
init_ap_data();
|
||||
#if !__USE_TBB_ATOMICS
|
||||
int res;
|
||||
res = pthread_spin_init( &handles.my_lock, PTHREAD_PROCESS_SHARED );
|
||||
LIBRARY_ASSERT( res==0, "pthread_spin_init failed" );
|
||||
#endif
|
||||
}
|
||||
|
||||
// ap_data structure is initialized with current directory on Linux.
|
||||
// So it should be initialized as soon as possible since the current directory may be changed.
|
||||
// static_init_ap_data object provides this initialization during library loading.
|
||||
static class _static_init_dl_data {
|
||||
public:
|
||||
_static_init_dl_data() {
|
||||
#if __USE_STATIC_DL_INIT
|
||||
atomic_once( &init_dl_data, init_dl_data_state );
|
||||
#endif
|
||||
}
|
||||
#if !__USE_TBB_ATOMICS
|
||||
~_static_init_dl_data() {
|
||||
int res;
|
||||
res = pthread_spin_destroy( &handles.my_lock );
|
||||
LIBRARY_ASSERT( res==0, "pthread_spin_destroy failed" );
|
||||
}
|
||||
#endif
|
||||
} static_init_dl_data;
|
||||
|
||||
/*
|
||||
The function constructs absolute path for given relative path. Important: Base directory is not
|
||||
current one, it is the directory libtbb.so loaded from.
|
||||
|
||||
Arguments:
|
||||
in name -- Name of a file (may be with relative path; it must not be an absolute one).
|
||||
out path -- Buffer to save result (absolute path) to.
|
||||
in len -- Size of buffer.
|
||||
ret -- 0 -- Error occurred.
|
||||
> len -- Buffer too short, required size returned.
|
||||
otherwise -- Ok, number of characters (not counting terminating null) written to
|
||||
buffer.
|
||||
*/
|
||||
#if __TBB_DYNAMIC_LOAD_ENABLED
|
||||
static size_t abs_path( char const * name, char * path, size_t len ) {
|
||||
atomic_once( &init_dl_data, init_dl_data_state );
|
||||
|
||||
if ( !ap_data._len )
|
||||
return 0;
|
||||
|
||||
size_t name_len = strlen( name );
|
||||
size_t full_len = name_len+ap_data._len;
|
||||
if ( full_len < len ) {
|
||||
strncpy( path, ap_data._path, ap_data._len );
|
||||
strncpy( path+ap_data._len, name, name_len );
|
||||
path[full_len] = 0;
|
||||
}
|
||||
return full_len;
|
||||
}
|
||||
#endif // __TBB_DYNAMIC_LOAD_ENABLED
|
||||
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
static bool weak_symbol_link( const dynamic_link_descriptor descriptors[], size_t required )
|
||||
{
|
||||
// Check if the required entries are present in what was loaded into our process.
|
||||
for ( size_t k = 0; k < required; ++k )
|
||||
if ( !descriptors[k].ptr )
|
||||
return false;
|
||||
// Commit the entry points.
|
||||
for ( size_t k = 0; k < required; ++k )
|
||||
*descriptors[k].handler = (pointer_to_handler) descriptors[k].ptr;
|
||||
return true;
|
||||
}
|
||||
#else
|
||||
static bool weak_symbol_link( const dynamic_link_descriptor[], size_t ) {
|
||||
return false;
|
||||
}
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
|
||||
void dynamic_unlink( dynamic_link_handle handle ) {
|
||||
if ( handle ) {
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
LIBRARY_ASSERT( dlclose != NULL, "dlopen is present but dlclose is NOT present!?" );
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
#if __TBB_DYNAMIC_LOAD_ENABLED
|
||||
dlclose( handle );
|
||||
#endif /* __TBB_DYNAMIC_LOAD_ENABLED */
|
||||
}
|
||||
}
|
||||
|
||||
void dynamic_unlink_all() {
|
||||
handles.free_handles();
|
||||
}
|
||||
|
||||
#if _WIN32
|
||||
static dynamic_link_handle global_symbols_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required ) {
|
||||
dynamic_link_handle library_handle;
|
||||
if ( GetModuleHandleExA( 0, library, &library_handle ) ) {
|
||||
if ( resolve_symbols( library_handle, descriptors, required ) )
|
||||
return library_handle;
|
||||
else
|
||||
FreeLibrary( library_handle );
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
#else /* _WIN32 */
|
||||
// It is supposed that all symbols are from the only one library
|
||||
static dynamic_link_handle pin_symbols( dynamic_link_descriptor desc, const dynamic_link_descriptor descriptors[], size_t required ) {
|
||||
// The library has been loaded by another module and contains at least one requested symbol.
|
||||
// But after we obtained the symbol the library can be unloaded by another thread
|
||||
// invalidating our symbol. Therefore we need to pin the library in memory.
|
||||
dynamic_link_handle library_handle;
|
||||
Dl_info info;
|
||||
// Get library's name from earlier found symbol
|
||||
if ( dladdr( (void*)*desc.handler, &info ) ) {
|
||||
// Pin the library
|
||||
library_handle = dlopen( info.dli_fname, RTLD_LAZY );
|
||||
if ( library_handle ) {
|
||||
// If original library was unloaded before we pinned it
|
||||
// and then another module loaded in its place, the earlier
|
||||
// found symbol would become invalid. So revalidate them.
|
||||
if ( !resolve_symbols( library_handle, descriptors, required ) ) {
|
||||
// Wrong library.
|
||||
dynamic_unlink(library_handle);
|
||||
library_handle = 0;
|
||||
}
|
||||
} else {
|
||||
char const * err = dlerror();
|
||||
DYNAMIC_LINK_WARNING( dl_lib_not_found, info.dli_fname, err );
|
||||
}
|
||||
}
|
||||
else {
|
||||
// The library have been unloaded by another thread
|
||||
library_handle = 0;
|
||||
}
|
||||
return library_handle;
|
||||
}
|
||||
|
||||
static dynamic_link_handle global_symbols_link( const char*, const dynamic_link_descriptor descriptors[], size_t required ) {
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
if ( !dlopen ) return 0;
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
dynamic_link_handle library_handle = dlopen( NULL, RTLD_LAZY );
|
||||
#if __ANDROID__
|
||||
// On Android dlopen( NULL ) returns NULL if it is called during dynamic module initialization.
|
||||
if ( !library_handle )
|
||||
return 0;
|
||||
#endif
|
||||
// Check existence of only the first symbol, then use it to find the library and load all necessary symbols
|
||||
pointer_to_handler handler;
|
||||
dynamic_link_descriptor desc = { descriptors[0].name, &handler };
|
||||
if ( resolve_symbols( library_handle, &desc, 1 ) )
|
||||
return pin_symbols( desc, descriptors, required );
|
||||
return 0;
|
||||
}
|
||||
#endif /* _WIN32 */
|
||||
|
||||
static void save_library_handle( dynamic_link_handle src, dynamic_link_handle *dst ) {
|
||||
if ( dst )
|
||||
*dst = src;
|
||||
else
|
||||
handles.add_handle( src );
|
||||
}
|
||||
|
||||
dynamic_link_handle dynamic_load( const char* library, const dynamic_link_descriptor descriptors[], size_t required ) {
|
||||
#if __TBB_DYNAMIC_LOAD_ENABLED
|
||||
#if _XBOX
|
||||
return LoadLibrary (library);
|
||||
#else /* _XBOX */
|
||||
size_t const len = PATH_MAX + 1;
|
||||
char path[ len ];
|
||||
size_t rc = abs_path( library, path, len );
|
||||
if ( 0 < rc && rc < len ) {
|
||||
#if _WIN32
|
||||
// Prevent Windows from displaying silly message boxes if it fails to load library
|
||||
// (e.g. because of MS runtime problems - one of those crazy manifest related ones)
|
||||
UINT prev_mode = SetErrorMode (SEM_FAILCRITICALERRORS);
|
||||
#endif /* _WIN32 */
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
if ( !dlopen ) return 0;
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
dynamic_link_handle library_handle = dlopen( path, RTLD_LAZY );
|
||||
#if _WIN32
|
||||
SetErrorMode (prev_mode);
|
||||
#endif /* _WIN32 */
|
||||
if( library_handle ) {
|
||||
if( !resolve_symbols( library_handle, descriptors, required ) ) {
|
||||
// The loaded library does not contain all the expected entry points
|
||||
dynamic_unlink( library_handle );
|
||||
library_handle = NULL;
|
||||
}
|
||||
} else
|
||||
DYNAMIC_LINK_WARNING( dl_lib_not_found, path, dlerror() );
|
||||
return library_handle;
|
||||
} else if ( rc>=len )
|
||||
DYNAMIC_LINK_WARNING( dl_buff_too_small );
|
||||
// rc == 0 means failing of init_ap_data so the warning has already been issued.
|
||||
#endif /* _XBOX */
|
||||
#endif /* __TBB_DYNAMIC_LOAD_ENABLED */
|
||||
return 0;
|
||||
}
|
||||
|
||||
bool dynamic_link( const char* library, const dynamic_link_descriptor descriptors[], size_t required, dynamic_link_handle *handle, int flags ) {
|
||||
// TODO: May global_symbols_link find weak symbols?
|
||||
dynamic_link_handle library_handle = ( flags & DYNAMIC_LINK_GLOBAL ) ? global_symbols_link( library, descriptors, required ) : 0;
|
||||
|
||||
if ( !library_handle && ( flags & DYNAMIC_LINK_LOAD ) )
|
||||
library_handle = dynamic_load( library, descriptors, required );
|
||||
|
||||
if ( !library_handle && ( flags & DYNAMIC_LINK_WEAK ) )
|
||||
return weak_symbol_link( descriptors, required );
|
||||
|
||||
save_library_handle( library_handle, handle );
|
||||
return true;
|
||||
}
|
||||
|
||||
#endif /*__TBB_WIN8UI_SUPPORT*/
|
||||
#else /* __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED */
|
||||
bool dynamic_link( const char*, const dynamic_link_descriptor*, size_t, dynamic_link_handle *handle, int ) {
|
||||
if ( handle )
|
||||
*handle=0;
|
||||
return false;
|
||||
}
|
||||
|
||||
void dynamic_unlink( dynamic_link_handle ) {
|
||||
}
|
||||
|
||||
void dynamic_unlink_all() {
|
||||
}
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT || __TBB_DYNAMIC_LOAD_ENABLED */
|
||||
|
||||
CLOSE_INTERNAL_NAMESPACE
|
||||
@@ -0,0 +1,121 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_dynamic_link
|
||||
#define __TBB_dynamic_link
|
||||
|
||||
// Support for dynamic loading entry points from other shared libraries.
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
|
||||
#ifdef LIBRARY_ASSERT
|
||||
#undef __TBB_ASSERT
|
||||
#define __TBB_ASSERT(x,y) LIBRARY_ASSERT(x,y)
|
||||
#else
|
||||
#define LIBRARY_ASSERT(x,y) __TBB_ASSERT_EX(x,y)
|
||||
#endif /* !LIBRARY_ASSERT */
|
||||
|
||||
/** By default, symbols declared and defined here go into namespace tbb::internal.
|
||||
To put them in other namespace, define macros OPEN_INTERNAL_NAMESPACE
|
||||
and CLOSE_INTERNAL_NAMESPACE to override the following default definitions. **/
|
||||
#ifndef OPEN_INTERNAL_NAMESPACE
|
||||
#define OPEN_INTERNAL_NAMESPACE namespace tbb { namespace internal {
|
||||
#define CLOSE_INTERNAL_NAMESPACE }}
|
||||
#endif /* OPEN_INTERNAL_NAMESPACE */
|
||||
|
||||
#include <stddef.h>
|
||||
#if _WIN32
|
||||
#include "tbb/machine/windows_api.h"
|
||||
#endif /* _WIN32 */
|
||||
|
||||
OPEN_INTERNAL_NAMESPACE
|
||||
|
||||
//! Type definition for a pointer to a void somefunc(void)
|
||||
typedef void (*pointer_to_handler)();
|
||||
|
||||
//! The helper to construct dynamic_link_descriptor structure
|
||||
// Double cast through the void* in DLD macro is necessary to
|
||||
// prevent warnings from some compilers (g++ 4.1)
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
#define DLD(s,h) {#s, (pointer_to_handler*)(void*)(&h), (pointer_to_handler)&s}
|
||||
#else
|
||||
#define DLD(s,h) {#s, (pointer_to_handler*)(void*)(&h)}
|
||||
#endif /* __TBB_WEAK_SYMBOLS_PRESENT */
|
||||
//! Association between a handler name and location of pointer to it.
|
||||
struct dynamic_link_descriptor {
|
||||
//! Name of the handler
|
||||
const char* name;
|
||||
//! Pointer to the handler
|
||||
pointer_to_handler* handler;
|
||||
#if __TBB_WEAK_SYMBOLS_PRESENT
|
||||
//! Weak symbol
|
||||
pointer_to_handler ptr;
|
||||
#endif
|
||||
};
|
||||
|
||||
#if _WIN32
|
||||
typedef HMODULE dynamic_link_handle;
|
||||
#else
|
||||
typedef void* dynamic_link_handle;
|
||||
#endif /* _WIN32 */
|
||||
|
||||
const int DYNAMIC_LINK_GLOBAL = 0x01;
|
||||
const int DYNAMIC_LINK_LOAD = 0x02;
|
||||
const int DYNAMIC_LINK_WEAK = 0x04;
|
||||
const int DYNAMIC_LINK_ALL = DYNAMIC_LINK_GLOBAL | DYNAMIC_LINK_LOAD | DYNAMIC_LINK_WEAK;
|
||||
|
||||
//! Fill in dynamically linked handlers.
|
||||
/** 'library' is the name of the requested library. It should not contain a full
|
||||
path since dynamic_link adds the full path (from which the runtime itself
|
||||
was loaded) to the library name.
|
||||
'required' is the number of the initial entries in the array descriptors[]
|
||||
that have to be found in order for the call to succeed. If the library and
|
||||
all the required handlers are found, then the corresponding handler
|
||||
pointers are set, and the return value is true. Otherwise the original
|
||||
array of descriptors is left untouched and the return value is false.
|
||||
'required' is limited by 20 (exceeding of this value will result in failure
|
||||
to load the symbols and the return value will be false).
|
||||
'handle' is the handle of the library if it is loaded. Otherwise it is left
|
||||
untouched.
|
||||
'flags' is the set of DYNAMIC_LINK_* flags. Each of the DYNAMIC_LINK_* flags
|
||||
allows its corresponding linking stage.
|
||||
**/
|
||||
bool dynamic_link( const char* library,
|
||||
const dynamic_link_descriptor descriptors[],
|
||||
size_t required,
|
||||
dynamic_link_handle* handle = 0,
|
||||
int flags = DYNAMIC_LINK_ALL );
|
||||
|
||||
void dynamic_unlink( dynamic_link_handle handle );
|
||||
|
||||
void dynamic_unlink_all();
|
||||
|
||||
enum dynamic_link_error_t {
|
||||
dl_success = 0,
|
||||
dl_lib_not_found, // char const * lib, dlerr_t err
|
||||
dl_sym_not_found, // char const * sym, dlerr_t err
|
||||
// Note: dlerr_t depends on OS: it is char const * on Linux* and OS X*, int on Windows*.
|
||||
dl_sys_fail, // char const * func, int err
|
||||
dl_buff_too_small // none
|
||||
}; // dynamic_link_error_t
|
||||
|
||||
CLOSE_INTERNAL_NAMESPACE
|
||||
|
||||
#endif /* __TBB_dynamic_link */
|
||||
File diff suppressed because it is too large
Load Diff
3237
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/flow_graph.h
vendored
Normal file
3237
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/flow_graph.h
vendored
Normal file
File diff suppressed because it is too large
Load Diff
346
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/governor.cpp
vendored
Normal file
346
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/governor.cpp
vendored
Normal file
@@ -0,0 +1,346 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include "governor.h"
|
||||
#include "tbb_main.h"
|
||||
#include "scheduler.h"
|
||||
#include "market.h"
|
||||
#include "arena.h"
|
||||
|
||||
#include "tbb/task_scheduler_init.h"
|
||||
|
||||
#include "dynamic_link.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// governor
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
// Support for interoperability with Intel(R) Cilk(TM) Plus.
|
||||
|
||||
#if _WIN32
|
||||
#define CILKLIB_NAME "cilkrts20.dll"
|
||||
#else
|
||||
#define CILKLIB_NAME "libcilkrts.so"
|
||||
#endif
|
||||
|
||||
//! Handler for interoperation with cilkrts library.
|
||||
static __cilk_tbb_retcode (*watch_stack_handler)(struct __cilk_tbb_unwatch_thunk* u,
|
||||
struct __cilk_tbb_stack_op_thunk o);
|
||||
|
||||
//! Table describing how to link the handlers.
|
||||
static const dynamic_link_descriptor CilkLinkTable[] = {
|
||||
{ "__cilkrts_watch_stack", (pointer_to_handler*)(void*)(&watch_stack_handler) }
|
||||
};
|
||||
|
||||
static atomic<do_once_state> cilkrts_load_state;
|
||||
|
||||
bool initialize_cilk_interop() {
|
||||
// Pinning can fail. This is a normal situation, and means that the current
|
||||
// thread does not use cilkrts and consequently does not need interop.
|
||||
return dynamic_link( CILKLIB_NAME, CilkLinkTable, 1, /*handle=*/0, DYNAMIC_LINK_GLOBAL );
|
||||
}
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
|
||||
namespace rml {
|
||||
tbb_server* make_private_server( tbb_client& client );
|
||||
}
|
||||
|
||||
void governor::acquire_resources () {
|
||||
#if USE_PTHREAD
|
||||
int status = theTLS.create(auto_terminate);
|
||||
#else
|
||||
int status = theTLS.create();
|
||||
#endif
|
||||
if( status )
|
||||
handle_perror(status, "TBB failed to initialize task scheduler TLS\n");
|
||||
is_speculation_enabled = cpu_has_speculation();
|
||||
}
|
||||
|
||||
void governor::release_resources () {
|
||||
theRMLServerFactory.close();
|
||||
#if TBB_USE_ASSERT
|
||||
if( __TBB_InitOnce::initialization_done() && theTLS.get() )
|
||||
runtime_warning( "TBB is unloaded while tbb::task_scheduler_init object is alive?" );
|
||||
#endif
|
||||
int status = theTLS.destroy();
|
||||
if( status )
|
||||
runtime_warning("failed to destroy task scheduler TLS: %s", strerror(status));
|
||||
dynamic_unlink_all();
|
||||
}
|
||||
|
||||
rml::tbb_server* governor::create_rml_server ( rml::tbb_client& client ) {
|
||||
rml::tbb_server* server = NULL;
|
||||
if( !UsePrivateRML ) {
|
||||
::rml::factory::status_type status = theRMLServerFactory.make_server( server, client );
|
||||
if( status != ::rml::factory::st_success ) {
|
||||
UsePrivateRML = true;
|
||||
runtime_warning( "rml::tbb_factory::make_server failed with status %x, falling back on private rml", status );
|
||||
}
|
||||
}
|
||||
if ( !server ) {
|
||||
__TBB_ASSERT( UsePrivateRML, NULL );
|
||||
server = rml::make_private_server( client );
|
||||
}
|
||||
__TBB_ASSERT( server, "Failed to create RML server" );
|
||||
return server;
|
||||
}
|
||||
|
||||
void governor::sign_on(generic_scheduler* s) {
|
||||
__TBB_ASSERT( !theTLS.get(), NULL );
|
||||
theTLS.set(s);
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
if( watch_stack_handler ) {
|
||||
__cilk_tbb_stack_op_thunk o;
|
||||
o.routine = &stack_op_handler;
|
||||
o.data = s;
|
||||
if( (*watch_stack_handler)(&s->my_cilk_unwatch_thunk, o) ) {
|
||||
// Failed to register with cilkrts, make sure we are clean
|
||||
s->my_cilk_unwatch_thunk.routine = NULL;
|
||||
}
|
||||
#if TBB_USE_ASSERT
|
||||
else
|
||||
s->my_cilk_state = generic_scheduler::cs_running;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
}
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
}
|
||||
|
||||
void governor::sign_off(generic_scheduler* s) {
|
||||
suppress_unused_warning(s);
|
||||
__TBB_ASSERT( theTLS.get()==s, "attempt to unregister a wrong scheduler instance" );
|
||||
theTLS.set(NULL);
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
__cilk_tbb_unwatch_thunk &ut = s->my_cilk_unwatch_thunk;
|
||||
if ( ut.routine )
|
||||
(*ut.routine)(ut.data);
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
}
|
||||
|
||||
void governor::setBlockingTerminate(const task_scheduler_init *tsi) {
|
||||
__TBB_ASSERT(!IsBlockingTerminationInProgress, "It's impossible to create task_scheduler_init while blocking termination is in progress.");
|
||||
if (BlockingTSI)
|
||||
throw_exception(eid_blocking_sch_init);
|
||||
BlockingTSI = tsi;
|
||||
}
|
||||
|
||||
generic_scheduler* governor::init_scheduler( unsigned num_threads, stack_size_type stack_size, bool auto_init ) {
|
||||
if( !__TBB_InitOnce::initialization_done() )
|
||||
DoOneTimeInitializations();
|
||||
generic_scheduler* s = theTLS.get();
|
||||
if( s ) {
|
||||
s->my_ref_count += 1;
|
||||
return s;
|
||||
}
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
atomic_do_once( &initialize_cilk_interop, cilkrts_load_state );
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
if( (int)num_threads == task_scheduler_init::automatic )
|
||||
num_threads = default_num_threads();
|
||||
s = generic_scheduler::create_master(
|
||||
market::create_arena( num_threads - 1, stack_size ? stack_size : ThreadStackSize ) );
|
||||
__TBB_ASSERT(s, "Somehow a local scheduler creation for a master thread failed");
|
||||
s->my_auto_initialized = auto_init;
|
||||
return s;
|
||||
}
|
||||
|
||||
void governor::terminate_scheduler( generic_scheduler* s, const task_scheduler_init* tsi_ptr ) {
|
||||
__TBB_ASSERT( s == theTLS.get(), "Attempt to terminate non-local scheduler instance" );
|
||||
if (--(s->my_ref_count)) {
|
||||
if (BlockingTSI && BlockingTSI==tsi_ptr) {
|
||||
// can't throw exception, because this is on dtor's call chain
|
||||
fprintf(stderr, "Attempt to terminate nested scheduler in blocking mode\n");
|
||||
exit(1);
|
||||
}
|
||||
} else {
|
||||
#if TBB_USE_ASSERT
|
||||
if (BlockingTSI) {
|
||||
__TBB_ASSERT( BlockingTSI == tsi_ptr, "For blocking termination last terminate_scheduler must be blocking." );
|
||||
IsBlockingTerminationInProgress = true;
|
||||
}
|
||||
#endif
|
||||
s->cleanup_master();
|
||||
BlockingTSI = NULL;
|
||||
#if TBB_USE_ASSERT
|
||||
IsBlockingTerminationInProgress = false;
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
void governor::auto_terminate(void* arg){
|
||||
generic_scheduler* s = static_cast<generic_scheduler*>(arg);
|
||||
if( s && s->my_auto_initialized ) {
|
||||
if( !--(s->my_ref_count) ) {
|
||||
__TBB_ASSERT( !BlockingTSI, "Blocking auto-terminate is not supported." );
|
||||
// If the TLS slot is already cleared by OS or underlying concurrency
|
||||
// runtime, restore its value.
|
||||
if ( !theTLS.get() )
|
||||
theTLS.set(s);
|
||||
else __TBB_ASSERT( s == theTLS.get(), NULL );
|
||||
s->cleanup_master();
|
||||
__TBB_ASSERT( !theTLS.get(), "cleanup_master has not cleared its TLS slot" );
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void governor::print_version_info () {
|
||||
if ( UsePrivateRML )
|
||||
PrintExtraVersionInfo( "RML", "private" );
|
||||
else {
|
||||
PrintExtraVersionInfo( "RML", "shared" );
|
||||
theRMLServerFactory.call_with_server_info( PrintRMLVersionInfo, (void*)"" );
|
||||
}
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
if( watch_stack_handler )
|
||||
PrintExtraVersionInfo( "CILK", CILKLIB_NAME );
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
}
|
||||
|
||||
void governor::initialize_rml_factory () {
|
||||
::rml::factory::status_type res = theRMLServerFactory.open();
|
||||
UsePrivateRML = res != ::rml::factory::st_success;
|
||||
}
|
||||
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
__cilk_tbb_retcode governor::stack_op_handler( __cilk_tbb_stack_op op, void* data ) {
|
||||
__TBB_ASSERT(data,NULL);
|
||||
generic_scheduler* s = static_cast<generic_scheduler*>(data);
|
||||
#if TBB_USE_ASSERT
|
||||
void* current = theTLS.get();
|
||||
#if _WIN32||_WIN64
|
||||
uintptr_t thread_id = GetCurrentThreadId();
|
||||
#else
|
||||
uintptr_t thread_id = uintptr_t(pthread_self());
|
||||
#endif
|
||||
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
switch( op ) {
|
||||
default:
|
||||
__TBB_ASSERT( 0, "invalid op" );
|
||||
case CILK_TBB_STACK_ADOPT: {
|
||||
__TBB_ASSERT( !current && s->my_cilk_state==generic_scheduler::cs_limbo ||
|
||||
current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid adoption" );
|
||||
#if TBB_USE_ASSERT
|
||||
if( current==s )
|
||||
runtime_warning( "redundant adoption of %p by thread %p\n", s, (void*)thread_id );
|
||||
s->my_cilk_state = generic_scheduler::cs_running;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
theTLS.set(s);
|
||||
break;
|
||||
}
|
||||
case CILK_TBB_STACK_ORPHAN: {
|
||||
__TBB_ASSERT( current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid orphaning" );
|
||||
#if TBB_USE_ASSERT
|
||||
s->my_cilk_state = generic_scheduler::cs_limbo;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
theTLS.set(NULL);
|
||||
break;
|
||||
}
|
||||
case CILK_TBB_STACK_RELEASE: {
|
||||
__TBB_ASSERT( !current && s->my_cilk_state==generic_scheduler::cs_limbo ||
|
||||
current==s && s->my_cilk_state==generic_scheduler::cs_running, "invalid release" );
|
||||
#if TBB_USE_ASSERT
|
||||
s->my_cilk_state = generic_scheduler::cs_freed;
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
s->my_cilk_unwatch_thunk.routine = NULL;
|
||||
auto_terminate( s );
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
|
||||
} // namespace internal
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// task_scheduler_init
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
using namespace internal;
|
||||
|
||||
/** Left out-of-line for the sake of the backward binary compatibility **/
|
||||
void task_scheduler_init::initialize( int number_of_threads ) {
|
||||
initialize( number_of_threads, 0 );
|
||||
}
|
||||
|
||||
void task_scheduler_init::initialize( int number_of_threads, stack_size_type thread_stack_size ) {
|
||||
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
|
||||
uintptr_t new_mode = thread_stack_size & propagation_mode_mask;
|
||||
#endif
|
||||
thread_stack_size &= ~(stack_size_type)propagation_mode_mask;
|
||||
if( number_of_threads!=deferred ) {
|
||||
bool blocking_terminate = false;
|
||||
if (my_scheduler == (scheduler*)wait_workers_in_terminate_flag) {
|
||||
blocking_terminate = true;
|
||||
my_scheduler = NULL;
|
||||
}
|
||||
__TBB_ASSERT( !my_scheduler, "task_scheduler_init already initialized" );
|
||||
__TBB_ASSERT( number_of_threads==-1 || number_of_threads>=1,
|
||||
"number_of_threads for task_scheduler_init must be -1 or positive" );
|
||||
if (blocking_terminate)
|
||||
governor::setBlockingTerminate(this);
|
||||
internal::generic_scheduler *s = governor::init_scheduler( number_of_threads, thread_stack_size, /*auto_init=*/false );
|
||||
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
|
||||
if ( s->master_outermost_level() ) {
|
||||
uintptr_t &vt = s->default_context()->my_version_and_traits;
|
||||
uintptr_t prev_mode = vt & task_group_context::exact_exception ? propagation_mode_exact : 0;
|
||||
vt = new_mode & propagation_mode_exact ? vt | task_group_context::exact_exception
|
||||
: new_mode & propagation_mode_captured ? vt & ~task_group_context::exact_exception : vt;
|
||||
// Use least significant bit of the scheduler pointer to store previous mode.
|
||||
// This is necessary when components compiled with different compilers and/or
|
||||
// TBB versions initialize the
|
||||
my_scheduler = static_cast<scheduler*>((generic_scheduler*)((uintptr_t)s | prev_mode));
|
||||
}
|
||||
else
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
|
||||
my_scheduler = s;
|
||||
} else {
|
||||
__TBB_ASSERT( !thread_stack_size, "deferred initialization ignores stack size setting" );
|
||||
}
|
||||
}
|
||||
|
||||
void task_scheduler_init::terminate() {
|
||||
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
|
||||
uintptr_t prev_mode = (uintptr_t)my_scheduler & propagation_mode_exact;
|
||||
my_scheduler = (scheduler*)((uintptr_t)my_scheduler & ~(uintptr_t)propagation_mode_exact);
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
|
||||
generic_scheduler* s = static_cast<generic_scheduler*>(my_scheduler);
|
||||
my_scheduler = NULL;
|
||||
__TBB_ASSERT( s, "task_scheduler_init::terminate without corresponding task_scheduler_init::initialize()");
|
||||
#if __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS
|
||||
if ( s->master_outermost_level() ) {
|
||||
uintptr_t &vt = s->default_context()->my_version_and_traits;
|
||||
vt = prev_mode & propagation_mode_exact ? vt | task_group_context::exact_exception
|
||||
: vt & ~task_group_context::exact_exception;
|
||||
}
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT && TBB_USE_EXCEPTIONS */
|
||||
governor::terminate_scheduler(s, this);
|
||||
}
|
||||
|
||||
int task_scheduler_init::default_num_threads() {
|
||||
return governor::default_num_threads();
|
||||
}
|
||||
|
||||
} // namespace tbb
|
||||
146
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/governor.h
vendored
Normal file
146
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/governor.h
vendored
Normal file
@@ -0,0 +1,146 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_governor_H
|
||||
#define _TBB_governor_H
|
||||
|
||||
#include "tbb/task_scheduler_init.h"
|
||||
#include "../rml/include/rml_tbb.h"
|
||||
|
||||
#include "tbb_misc.h" // for AvailableHwConcurrency and ThreadStackSize
|
||||
#include "tls.h"
|
||||
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
#include "cilk-tbb-interop.h"
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
class market;
|
||||
class generic_scheduler;
|
||||
class __TBB_InitOnce;
|
||||
|
||||
//------------------------------------------------------------------------
|
||||
// Class governor
|
||||
//------------------------------------------------------------------------
|
||||
|
||||
//! The class handles access to the single instance of market, and to TLS to keep scheduler instances.
|
||||
/** It also supports automatic on-demand initialization of the TBB scheduler.
|
||||
The class contains only static data members and methods.*/
|
||||
class governor {
|
||||
friend class __TBB_InitOnce;
|
||||
friend class market;
|
||||
|
||||
//! TLS for scheduler instances associated with individual threads
|
||||
static basic_tls<generic_scheduler*> theTLS;
|
||||
|
||||
//! Caches the maximal level of parallelism supported by the hardware
|
||||
static unsigned DefaultNumberOfThreads;
|
||||
|
||||
static rml::tbb_factory theRMLServerFactory;
|
||||
|
||||
static bool UsePrivateRML;
|
||||
|
||||
//! Instance of task_scheduler_init that requested blocking termination.
|
||||
static const task_scheduler_init *BlockingTSI;
|
||||
|
||||
#if TBB_USE_ASSERT
|
||||
static bool IsBlockingTerminationInProgress;
|
||||
#endif
|
||||
|
||||
static bool is_speculation_enabled;
|
||||
|
||||
//! Create key for thread-local storage and initialize RML.
|
||||
static void acquire_resources ();
|
||||
|
||||
//! Destroy the thread-local storage key and deinitialize RML.
|
||||
static void release_resources ();
|
||||
|
||||
static rml::tbb_server* create_rml_server ( rml::tbb_client& );
|
||||
|
||||
//! The internal routine to undo automatic initialization.
|
||||
/** The signature is written with void* so that the routine
|
||||
can be the destructor argument to pthread_key_create. */
|
||||
static void auto_terminate(void* scheduler);
|
||||
|
||||
public:
|
||||
static unsigned default_num_threads () {
|
||||
// No memory fence required, because at worst each invoking thread calls AvailableHwConcurrency once.
|
||||
return DefaultNumberOfThreads ? DefaultNumberOfThreads :
|
||||
DefaultNumberOfThreads = AvailableHwConcurrency();
|
||||
}
|
||||
//! Processes scheduler initialization request (possibly nested) in a master thread
|
||||
/** If necessary creates new instance of arena and/or local scheduler.
|
||||
The auto_init argument specifies if the call is due to automatic initialization. **/
|
||||
static generic_scheduler* init_scheduler( unsigned num_threads, stack_size_type stack_size, bool auto_init = false );
|
||||
|
||||
//! Processes scheduler termination request (possibly nested) in a master thread
|
||||
static void terminate_scheduler( generic_scheduler* s, const task_scheduler_init *tsi_ptr );
|
||||
|
||||
//! Register TBB scheduler instance in thread-local storage.
|
||||
static void sign_on(generic_scheduler* s);
|
||||
|
||||
//! Unregister TBB scheduler instance from thread-local storage.
|
||||
static void sign_off(generic_scheduler* s);
|
||||
|
||||
//! Used to check validity of the local scheduler TLS contents.
|
||||
static bool is_set ( generic_scheduler* s ) { return theTLS.get() == s; }
|
||||
|
||||
//! Temporarily set TLS slot to the given scheduler
|
||||
static void assume_scheduler( generic_scheduler* s ) { theTLS.set( s ); }
|
||||
|
||||
//! Obtain the thread-local instance of the TBB scheduler.
|
||||
/** If the scheduler has not been initialized yet, initialization is done automatically.
|
||||
Note that auto-initialized scheduler instance is destroyed only when its thread terminates. **/
|
||||
static generic_scheduler* local_scheduler () {
|
||||
generic_scheduler* s = theTLS.get();
|
||||
return s ? s : init_scheduler( (unsigned)task_scheduler_init::automatic, 0, true );
|
||||
}
|
||||
|
||||
static generic_scheduler* local_scheduler_if_initialized () {
|
||||
return theTLS.get();
|
||||
}
|
||||
|
||||
//! Undo automatic initialization if necessary; call when a thread exits.
|
||||
static void terminate_auto_initialized_scheduler() {
|
||||
auto_terminate( theTLS.get() );
|
||||
}
|
||||
|
||||
static void print_version_info ();
|
||||
|
||||
static void initialize_rml_factory ();
|
||||
|
||||
static bool needsWaitWorkers () { return BlockingTSI!=NULL; }
|
||||
|
||||
//! Must be called before init_scheduler
|
||||
static void setBlockingTerminate(const task_scheduler_init *tsi);
|
||||
|
||||
#if __TBB_SURVIVE_THREAD_SWITCH
|
||||
static __cilk_tbb_retcode stack_op_handler( __cilk_tbb_stack_op op, void* );
|
||||
#endif /* __TBB_SURVIVE_THREAD_SWITCH */
|
||||
static bool speculation_enabled() { return is_speculation_enabled; }
|
||||
|
||||
}; // class governor
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* _TBB_governor_H */
|
||||
@@ -0,0 +1,188 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.686
|
||||
.model flat,c
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchadd1
|
||||
__TBB_machine_fetchadd1:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xadd [edx],al
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchstore1
|
||||
__TBB_machine_fetchstore1:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xchg [edx],al
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_cmpswp1
|
||||
__TBB_machine_cmpswp1:
|
||||
mov edx,4[esp]
|
||||
mov ecx,8[esp]
|
||||
mov eax,12[esp]
|
||||
lock cmpxchg [edx],cl
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchadd2
|
||||
__TBB_machine_fetchadd2:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xadd [edx],ax
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchstore2
|
||||
__TBB_machine_fetchstore2:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xchg [edx],ax
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_cmpswp2
|
||||
__TBB_machine_cmpswp2:
|
||||
mov edx,4[esp]
|
||||
mov ecx,8[esp]
|
||||
mov eax,12[esp]
|
||||
lock cmpxchg [edx],cx
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchadd4
|
||||
__TBB_machine_fetchadd4:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xadd [edx],eax
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchstore4
|
||||
__TBB_machine_fetchstore4:
|
||||
mov edx,4[esp]
|
||||
mov eax,8[esp]
|
||||
lock xchg [edx],eax
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_cmpswp4
|
||||
__TBB_machine_cmpswp4:
|
||||
mov edx,4[esp]
|
||||
mov ecx,8[esp]
|
||||
mov eax,12[esp]
|
||||
lock cmpxchg [edx],ecx
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchadd8
|
||||
__TBB_machine_fetchadd8:
|
||||
push ebx
|
||||
push edi
|
||||
mov edi,12[esp]
|
||||
mov eax,[edi]
|
||||
mov edx,4[edi]
|
||||
__TBB_machine_fetchadd8_loop:
|
||||
mov ebx,16[esp]
|
||||
mov ecx,20[esp]
|
||||
add ebx,eax
|
||||
adc ecx,edx
|
||||
lock cmpxchg8b qword ptr [edi]
|
||||
jnz __TBB_machine_fetchadd8_loop
|
||||
pop edi
|
||||
pop ebx
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_fetchstore8
|
||||
__TBB_machine_fetchstore8:
|
||||
push ebx
|
||||
push edi
|
||||
mov edi,12[esp]
|
||||
mov ebx,16[esp]
|
||||
mov ecx,20[esp]
|
||||
mov eax,[edi]
|
||||
mov edx,4[edi]
|
||||
__TBB_machine_fetchstore8_loop:
|
||||
lock cmpxchg8b qword ptr [edi]
|
||||
jnz __TBB_machine_fetchstore8_loop
|
||||
pop edi
|
||||
pop ebx
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_cmpswp8
|
||||
__TBB_machine_cmpswp8:
|
||||
push ebx
|
||||
push edi
|
||||
mov edi,12[esp]
|
||||
mov ebx,16[esp]
|
||||
mov ecx,20[esp]
|
||||
mov eax,24[esp]
|
||||
mov edx,28[esp]
|
||||
lock cmpxchg8b qword ptr [edi]
|
||||
pop edi
|
||||
pop ebx
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_load8
|
||||
__TBB_machine_Load8:
|
||||
; If location is on stack, compiler may have failed to align it correctly, so we do dynamic check.
|
||||
mov ecx,4[esp]
|
||||
test ecx,7
|
||||
jne load_slow
|
||||
; Load within a cache line
|
||||
sub esp,12
|
||||
fild qword ptr [ecx]
|
||||
fistp qword ptr [esp]
|
||||
mov eax,[esp]
|
||||
mov edx,4[esp]
|
||||
add esp,12
|
||||
ret
|
||||
load_slow:
|
||||
; Load is misaligned. Use cmpxchg8b.
|
||||
push ebx
|
||||
push edi
|
||||
mov edi,ecx
|
||||
xor eax,eax
|
||||
xor ebx,ebx
|
||||
xor ecx,ecx
|
||||
xor edx,edx
|
||||
lock cmpxchg8b qword ptr [edi]
|
||||
pop edi
|
||||
pop ebx
|
||||
ret
|
||||
EXTRN __TBB_machine_store8_slow:PROC
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_store8
|
||||
__TBB_machine_Store8:
|
||||
; If location is on stack, compiler may have failed to align it correctly, so we do dynamic check.
|
||||
mov ecx,4[esp]
|
||||
test ecx,7
|
||||
jne __TBB_machine_store8_slow ;; tail call to tbb_misc.cpp
|
||||
fild qword ptr 8[esp]
|
||||
fistp qword ptr [ecx]
|
||||
ret
|
||||
end
|
||||
@@ -0,0 +1,80 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.686
|
||||
.model flat,c
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_try_lock_elided
|
||||
__TBB_machine_try_lock_elided:
|
||||
mov ecx, 4[esp]
|
||||
xor eax, eax
|
||||
mov al, 1
|
||||
BYTE 0F2H
|
||||
xchg al, byte ptr [ecx]
|
||||
xor al, 1
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_unlock_elided
|
||||
__TBB_machine_unlock_elided:
|
||||
mov ecx, 4[esp]
|
||||
BYTE 0F3H
|
||||
mov byte ptr [ecx], 0
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_begin_transaction
|
||||
__TBB_machine_begin_transaction:
|
||||
mov eax, -1
|
||||
BYTE 0C7H
|
||||
BYTE 0F8H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_end_transaction
|
||||
__TBB_machine_end_transaction:
|
||||
BYTE 00FH
|
||||
BYTE 001H
|
||||
BYTE 0D5H
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_transaction_conflict_abort
|
||||
__TBB_machine_transaction_conflict_abort:
|
||||
BYTE 0C6H
|
||||
BYTE 0F8H
|
||||
BYTE 0FFH ; 12.4.5 Abort argument: lock not free when tested
|
||||
ret
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_is_in_transaction
|
||||
__TBB_machine_is_in_transaction:
|
||||
xor eax, eax
|
||||
BYTE 00FH
|
||||
BYTE 001H
|
||||
BYTE 0D6H
|
||||
JZ rset
|
||||
MOV al,1
|
||||
rset:
|
||||
RET
|
||||
end
|
||||
@@ -0,0 +1,38 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
; DO NOT EDIT - AUTOMATICALLY GENERATED FROM .s FILE
|
||||
.686
|
||||
.model flat,c
|
||||
.code
|
||||
ALIGN 4
|
||||
PUBLIC c __TBB_machine_trylockbyte
|
||||
__TBB_machine_trylockbyte:
|
||||
mov edx,4[esp]
|
||||
mov al,[edx]
|
||||
mov cl,1
|
||||
test al,1
|
||||
jnz __TBB_machine_trylockbyte_contended
|
||||
lock cmpxchg [edx],cl
|
||||
jne __TBB_machine_trylockbyte_contended
|
||||
mov eax,1
|
||||
ret
|
||||
__TBB_machine_trylockbyte_contended:
|
||||
xor eax,eax
|
||||
ret
|
||||
end
|
||||
@@ -0,0 +1,670 @@
|
||||
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
//
|
||||
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
// you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
// See the GNU General Public License for more details. You should have received a copy of
|
||||
// the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
//
|
||||
// As a special exception, you may use this file as part of a free software library without
|
||||
// restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
// functions from this file, or you compile this file and link it with other files to produce
|
||||
// an executable, this file does not by itself cause the resulting executable to be covered
|
||||
// by the GNU General Public License. This exception does not however invalidate any other
|
||||
// reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchadd1__TBB_full_fence#
|
||||
.global __TBB_machine_fetchadd1__TBB_full_fence#
|
||||
__TBB_machine_fetchadd1__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_fetchadd1acquire
|
||||
}
|
||||
.endp __TBB_machine_fetchadd1__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_fetchadd1acquire#
|
||||
.global __TBB_machine_fetchadd1acquire#
|
||||
__TBB_machine_fetchadd1acquire:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
ld1 r9=[r32]
|
||||
;;
|
||||
Retry_1acquire:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg1.acq r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_1acquire
|
||||
br.ret.sptk.many b0
|
||||
# 49 "<stdin>"
|
||||
.endp __TBB_machine_fetchadd1acquire#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore1__TBB_full_fence#
|
||||
.global __TBB_machine_fetchstore1__TBB_full_fence#
|
||||
__TBB_machine_fetchstore1__TBB_full_fence:
|
||||
mf
|
||||
;;
|
||||
xchg1 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore1__TBB_full_fence#
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchstore1acquire#
|
||||
.global __TBB_machine_fetchstore1acquire#
|
||||
__TBB_machine_fetchstore1acquire:
|
||||
xchg1 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore1acquire#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_cmpswp1__TBB_full_fence#
|
||||
.global __TBB_machine_cmpswp1__TBB_full_fence#
|
||||
__TBB_machine_cmpswp1__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_cmpswp1acquire
|
||||
}
|
||||
.endp __TBB_machine_cmpswp1__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_cmpswp1acquire#
|
||||
.global __TBB_machine_cmpswp1acquire#
|
||||
__TBB_machine_cmpswp1acquire:
|
||||
|
||||
zxt1 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg1.acq r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp1acquire#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchadd2__TBB_full_fence#
|
||||
.global __TBB_machine_fetchadd2__TBB_full_fence#
|
||||
__TBB_machine_fetchadd2__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_fetchadd2acquire
|
||||
}
|
||||
.endp __TBB_machine_fetchadd2__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_fetchadd2acquire#
|
||||
.global __TBB_machine_fetchadd2acquire#
|
||||
__TBB_machine_fetchadd2acquire:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
ld2 r9=[r32]
|
||||
;;
|
||||
Retry_2acquire:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg2.acq r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_2acquire
|
||||
br.ret.sptk.many b0
|
||||
# 49 "<stdin>"
|
||||
.endp __TBB_machine_fetchadd2acquire#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore2__TBB_full_fence#
|
||||
.global __TBB_machine_fetchstore2__TBB_full_fence#
|
||||
__TBB_machine_fetchstore2__TBB_full_fence:
|
||||
mf
|
||||
;;
|
||||
xchg2 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore2__TBB_full_fence#
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchstore2acquire#
|
||||
.global __TBB_machine_fetchstore2acquire#
|
||||
__TBB_machine_fetchstore2acquire:
|
||||
xchg2 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore2acquire#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_cmpswp2__TBB_full_fence#
|
||||
.global __TBB_machine_cmpswp2__TBB_full_fence#
|
||||
__TBB_machine_cmpswp2__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_cmpswp2acquire
|
||||
}
|
||||
.endp __TBB_machine_cmpswp2__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_cmpswp2acquire#
|
||||
.global __TBB_machine_cmpswp2acquire#
|
||||
__TBB_machine_cmpswp2acquire:
|
||||
|
||||
zxt2 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg2.acq r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp2acquire#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchadd4__TBB_full_fence#
|
||||
.global __TBB_machine_fetchadd4__TBB_full_fence#
|
||||
__TBB_machine_fetchadd4__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_fetchadd4acquire
|
||||
}
|
||||
.endp __TBB_machine_fetchadd4__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_fetchadd4acquire#
|
||||
.global __TBB_machine_fetchadd4acquire#
|
||||
__TBB_machine_fetchadd4acquire:
|
||||
|
||||
cmp.eq p6,p0=1,r33
|
||||
cmp.eq p8,p0=-1,r33
|
||||
(p6) br.cond.dptk Inc_4acquire
|
||||
(p8) br.cond.dpnt Dec_4acquire
|
||||
;;
|
||||
|
||||
ld4 r9=[r32]
|
||||
;;
|
||||
Retry_4acquire:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg4.acq r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_4acquire
|
||||
br.ret.sptk.many b0
|
||||
|
||||
Inc_4acquire:
|
||||
fetchadd4.acq r8=[r32],1
|
||||
br.ret.sptk.many b0
|
||||
Dec_4acquire:
|
||||
fetchadd4.acq r8=[r32],-1
|
||||
br.ret.sptk.many b0
|
||||
|
||||
.endp __TBB_machine_fetchadd4acquire#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore4__TBB_full_fence#
|
||||
.global __TBB_machine_fetchstore4__TBB_full_fence#
|
||||
__TBB_machine_fetchstore4__TBB_full_fence:
|
||||
mf
|
||||
;;
|
||||
xchg4 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore4__TBB_full_fence#
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchstore4acquire#
|
||||
.global __TBB_machine_fetchstore4acquire#
|
||||
__TBB_machine_fetchstore4acquire:
|
||||
xchg4 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore4acquire#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_cmpswp4__TBB_full_fence#
|
||||
.global __TBB_machine_cmpswp4__TBB_full_fence#
|
||||
__TBB_machine_cmpswp4__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_cmpswp4acquire
|
||||
}
|
||||
.endp __TBB_machine_cmpswp4__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_cmpswp4acquire#
|
||||
.global __TBB_machine_cmpswp4acquire#
|
||||
__TBB_machine_cmpswp4acquire:
|
||||
|
||||
zxt4 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg4.acq r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp4acquire#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchadd8__TBB_full_fence#
|
||||
.global __TBB_machine_fetchadd8__TBB_full_fence#
|
||||
__TBB_machine_fetchadd8__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_fetchadd8acquire
|
||||
}
|
||||
.endp __TBB_machine_fetchadd8__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_fetchadd8acquire#
|
||||
.global __TBB_machine_fetchadd8acquire#
|
||||
__TBB_machine_fetchadd8acquire:
|
||||
|
||||
cmp.eq p6,p0=1,r33
|
||||
cmp.eq p8,p0=-1,r33
|
||||
(p6) br.cond.dptk Inc_8acquire
|
||||
(p8) br.cond.dpnt Dec_8acquire
|
||||
;;
|
||||
|
||||
ld8 r9=[r32]
|
||||
;;
|
||||
Retry_8acquire:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg8.acq r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_8acquire
|
||||
br.ret.sptk.many b0
|
||||
|
||||
Inc_8acquire:
|
||||
fetchadd8.acq r8=[r32],1
|
||||
br.ret.sptk.many b0
|
||||
Dec_8acquire:
|
||||
fetchadd8.acq r8=[r32],-1
|
||||
br.ret.sptk.many b0
|
||||
|
||||
.endp __TBB_machine_fetchadd8acquire#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore8__TBB_full_fence#
|
||||
.global __TBB_machine_fetchstore8__TBB_full_fence#
|
||||
__TBB_machine_fetchstore8__TBB_full_fence:
|
||||
mf
|
||||
;;
|
||||
xchg8 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore8__TBB_full_fence#
|
||||
|
||||
|
||||
.proc __TBB_machine_fetchstore8acquire#
|
||||
.global __TBB_machine_fetchstore8acquire#
|
||||
__TBB_machine_fetchstore8acquire:
|
||||
xchg8 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore8acquire#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
|
||||
|
||||
.proc __TBB_machine_cmpswp8__TBB_full_fence#
|
||||
.global __TBB_machine_cmpswp8__TBB_full_fence#
|
||||
__TBB_machine_cmpswp8__TBB_full_fence:
|
||||
{
|
||||
mf
|
||||
br __TBB_machine_cmpswp8acquire
|
||||
}
|
||||
.endp __TBB_machine_cmpswp8__TBB_full_fence#
|
||||
|
||||
.proc __TBB_machine_cmpswp8acquire#
|
||||
.global __TBB_machine_cmpswp8acquire#
|
||||
__TBB_machine_cmpswp8acquire:
|
||||
|
||||
|
||||
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg8.acq r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp8acquire#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
# 19 "<stdin>"
|
||||
.proc __TBB_machine_fetchadd1release#
|
||||
.global __TBB_machine_fetchadd1release#
|
||||
__TBB_machine_fetchadd1release:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
ld1 r9=[r32]
|
||||
;;
|
||||
Retry_1release:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg1.rel r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_1release
|
||||
br.ret.sptk.many b0
|
||||
# 49 "<stdin>"
|
||||
.endp __TBB_machine_fetchadd1release#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore1release#
|
||||
.global __TBB_machine_fetchstore1release#
|
||||
__TBB_machine_fetchstore1release:
|
||||
mf
|
||||
;;
|
||||
xchg1 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore1release#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
# 101 "<stdin>"
|
||||
.proc __TBB_machine_cmpswp1release#
|
||||
.global __TBB_machine_cmpswp1release#
|
||||
__TBB_machine_cmpswp1release:
|
||||
|
||||
zxt1 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg1.rel r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp1release#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
# 19 "<stdin>"
|
||||
.proc __TBB_machine_fetchadd2release#
|
||||
.global __TBB_machine_fetchadd2release#
|
||||
__TBB_machine_fetchadd2release:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
ld2 r9=[r32]
|
||||
;;
|
||||
Retry_2release:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg2.rel r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_2release
|
||||
br.ret.sptk.many b0
|
||||
# 49 "<stdin>"
|
||||
.endp __TBB_machine_fetchadd2release#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore2release#
|
||||
.global __TBB_machine_fetchstore2release#
|
||||
__TBB_machine_fetchstore2release:
|
||||
mf
|
||||
;;
|
||||
xchg2 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore2release#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
# 101 "<stdin>"
|
||||
.proc __TBB_machine_cmpswp2release#
|
||||
.global __TBB_machine_cmpswp2release#
|
||||
__TBB_machine_cmpswp2release:
|
||||
|
||||
zxt2 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg2.rel r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp2release#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
# 19 "<stdin>"
|
||||
.proc __TBB_machine_fetchadd4release#
|
||||
.global __TBB_machine_fetchadd4release#
|
||||
__TBB_machine_fetchadd4release:
|
||||
|
||||
cmp.eq p6,p0=1,r33
|
||||
cmp.eq p8,p0=-1,r33
|
||||
(p6) br.cond.dptk Inc_4release
|
||||
(p8) br.cond.dpnt Dec_4release
|
||||
;;
|
||||
|
||||
ld4 r9=[r32]
|
||||
;;
|
||||
Retry_4release:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg4.rel r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_4release
|
||||
br.ret.sptk.many b0
|
||||
|
||||
Inc_4release:
|
||||
fetchadd4.rel r8=[r32],1
|
||||
br.ret.sptk.many b0
|
||||
Dec_4release:
|
||||
fetchadd4.rel r8=[r32],-1
|
||||
br.ret.sptk.many b0
|
||||
|
||||
.endp __TBB_machine_fetchadd4release#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore4release#
|
||||
.global __TBB_machine_fetchstore4release#
|
||||
__TBB_machine_fetchstore4release:
|
||||
mf
|
||||
;;
|
||||
xchg4 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore4release#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
# 101 "<stdin>"
|
||||
.proc __TBB_machine_cmpswp4release#
|
||||
.global __TBB_machine_cmpswp4release#
|
||||
__TBB_machine_cmpswp4release:
|
||||
|
||||
zxt4 r34=r34
|
||||
;;
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg4.rel r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp4release#
|
||||
// DO NOT EDIT - AUTOMATICALLY GENERATED FROM tools/generate_atomic/ipf_generate.sh
|
||||
# 1 "<stdin>"
|
||||
# 1 "<built-in>"
|
||||
# 1 "<command line>"
|
||||
# 1 "<stdin>"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
# 19 "<stdin>"
|
||||
.proc __TBB_machine_fetchadd8release#
|
||||
.global __TBB_machine_fetchadd8release#
|
||||
__TBB_machine_fetchadd8release:
|
||||
|
||||
cmp.eq p6,p0=1,r33
|
||||
cmp.eq p8,p0=-1,r33
|
||||
(p6) br.cond.dptk Inc_8release
|
||||
(p8) br.cond.dpnt Dec_8release
|
||||
;;
|
||||
|
||||
ld8 r9=[r32]
|
||||
;;
|
||||
Retry_8release:
|
||||
mov ar.ccv=r9
|
||||
mov r8=r9;
|
||||
add r10=r9,r33
|
||||
;;
|
||||
cmpxchg8.rel r9=[r32],r10,ar.ccv
|
||||
;;
|
||||
cmp.ne p7,p0=r8,r9
|
||||
(p7) br.cond.dpnt Retry_8release
|
||||
br.ret.sptk.many b0
|
||||
|
||||
Inc_8release:
|
||||
fetchadd8.rel r8=[r32],1
|
||||
br.ret.sptk.many b0
|
||||
Dec_8release:
|
||||
fetchadd8.rel r8=[r32],-1
|
||||
br.ret.sptk.many b0
|
||||
|
||||
.endp __TBB_machine_fetchadd8release#
|
||||
# 62 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_fetchstore8release#
|
||||
.global __TBB_machine_fetchstore8release#
|
||||
__TBB_machine_fetchstore8release:
|
||||
mf
|
||||
;;
|
||||
xchg8 r8=[r32],r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_fetchstore8release#
|
||||
# 88 "<stdin>"
|
||||
.section .text
|
||||
.align 16
|
||||
# 101 "<stdin>"
|
||||
.proc __TBB_machine_cmpswp8release#
|
||||
.global __TBB_machine_cmpswp8release#
|
||||
__TBB_machine_cmpswp8release:
|
||||
|
||||
|
||||
|
||||
|
||||
mov ar.ccv=r34
|
||||
;;
|
||||
cmpxchg8.rel r8=[r32],r33,ar.ccv
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_cmpswp8release#
|
||||
@@ -0,0 +1,99 @@
|
||||
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
//
|
||||
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
// you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
// See the GNU General Public License for more details. You should have received a copy of
|
||||
// the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
//
|
||||
// As a special exception, you may use this file as part of a free software library without
|
||||
// restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
// functions from this file, or you compile this file and link it with other files to produce
|
||||
// an executable, this file does not by itself cause the resulting executable to be covered
|
||||
// by the GNU General Public License. This exception does not however invalidate any other
|
||||
// reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
// RSE backing store pointer retrieval
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_get_bsp#
|
||||
.global __TBB_get_bsp#
|
||||
__TBB_get_bsp:
|
||||
mov r8=ar.bsp
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_get_bsp#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_load8_relaxed#
|
||||
.global __TBB_machine_load8_relaxed#
|
||||
__TBB_machine_load8_relaxed:
|
||||
ld8 r8=[r32]
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_load8_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_store8_relaxed#
|
||||
.global __TBB_machine_store8_relaxed#
|
||||
__TBB_machine_store8_relaxed:
|
||||
st8 [r32]=r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_store8_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_load4_relaxed#
|
||||
.global __TBB_machine_load4_relaxed#
|
||||
__TBB_machine_load4_relaxed:
|
||||
ld4 r8=[r32]
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_load4_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_store4_relaxed#
|
||||
.global __TBB_machine_store4_relaxed#
|
||||
__TBB_machine_store4_relaxed:
|
||||
st4 [r32]=r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_store4_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_load2_relaxed#
|
||||
.global __TBB_machine_load2_relaxed#
|
||||
__TBB_machine_load2_relaxed:
|
||||
ld2 r8=[r32]
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_load2_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_store2_relaxed#
|
||||
.global __TBB_machine_store2_relaxed#
|
||||
__TBB_machine_store2_relaxed:
|
||||
st2 [r32]=r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_store2_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_load1_relaxed#
|
||||
.global __TBB_machine_load1_relaxed#
|
||||
__TBB_machine_load1_relaxed:
|
||||
ld1 r8=[r32]
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_load1_relaxed#
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
.proc __TBB_machine_store1_relaxed#
|
||||
.global __TBB_machine_store1_relaxed#
|
||||
__TBB_machine_store1_relaxed:
|
||||
st1 [r32]=r33
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_store1_relaxed#
|
||||
@@ -0,0 +1,46 @@
|
||||
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
//
|
||||
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
// you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
// See the GNU General Public License for more details. You should have received a copy of
|
||||
// the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
//
|
||||
// As a special exception, you may use this file as part of a free software library without
|
||||
// restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
// functions from this file, or you compile this file and link it with other files to produce
|
||||
// an executable, this file does not by itself cause the resulting executable to be covered
|
||||
// by the GNU General Public License. This exception does not however invalidate any other
|
||||
// reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
// Support for class TinyLock
|
||||
.section .text
|
||||
.align 16
|
||||
// unsigned int __TBB_machine_trylockbyte( byte& flag );
|
||||
// r32 = address of flag
|
||||
.proc __TBB_machine_trylockbyte#
|
||||
.global __TBB_machine_trylockbyte#
|
||||
ADDRESS_OF_FLAG=r32
|
||||
RETCODE=r8
|
||||
FLAG=r9
|
||||
BUSY=r10
|
||||
SCRATCH=r11
|
||||
__TBB_machine_trylockbyte:
|
||||
ld1.acq FLAG=[ADDRESS_OF_FLAG]
|
||||
mov BUSY=1
|
||||
mov RETCODE=0
|
||||
;;
|
||||
cmp.ne p6,p0=0,FLAG
|
||||
mov ar.ccv=r0
|
||||
(p6) br.ret.sptk.many b0
|
||||
;;
|
||||
cmpxchg1.acq SCRATCH=[ADDRESS_OF_FLAG],BUSY,ar.ccv // Try to acquire lock
|
||||
;;
|
||||
cmp.eq p6,p0=0,SCRATCH
|
||||
;;
|
||||
(p6) mov RETCODE=1
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_trylockbyte#
|
||||
@@ -0,0 +1,58 @@
|
||||
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
//
|
||||
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
// you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
// See the GNU General Public License for more details. You should have received a copy of
|
||||
// the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
//
|
||||
// As a special exception, you may use this file as part of a free software library without
|
||||
// restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
// functions from this file, or you compile this file and link it with other files to produce
|
||||
// an executable, this file does not by itself cause the resulting executable to be covered
|
||||
// by the GNU General Public License. This exception does not however invalidate any other
|
||||
// reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
// unsigned long __TBB_machine_lg( unsigned long x );
|
||||
// r32 = x
|
||||
.proc __TBB_machine_lg#
|
||||
.global __TBB_machine_lg#
|
||||
__TBB_machine_lg:
|
||||
shr r16=r32,1 // .x
|
||||
;;
|
||||
shr r17=r32,2 // ..x
|
||||
or r32=r32,r16 // xx
|
||||
;;
|
||||
shr r16=r32,3 // ...xx
|
||||
or r32=r32,r17 // xxx
|
||||
;;
|
||||
shr r17=r32,5 // .....xxx
|
||||
or r32=r32,r16 // xxxxx
|
||||
;;
|
||||
shr r16=r32,8 // ........xxxxx
|
||||
or r32=r32,r17 // xxxxxxxx
|
||||
;;
|
||||
shr r17=r32,13
|
||||
or r32=r32,r16 // 13x
|
||||
;;
|
||||
shr r16=r32,21
|
||||
or r32=r32,r17 // 21x
|
||||
;;
|
||||
shr r17=r32,34
|
||||
or r32=r32,r16 // 34x
|
||||
;;
|
||||
shr r16=r32,55
|
||||
or r32=r32,r17 // 55x
|
||||
;;
|
||||
or r32=r32,r16 // 64x
|
||||
;;
|
||||
popcnt r8=r32
|
||||
;;
|
||||
add r8=-1,r8
|
||||
br.ret.sptk.many b0
|
||||
.endp __TBB_machine_lg#
|
||||
@@ -0,0 +1,33 @@
|
||||
// Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
//
|
||||
// This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
// you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
// version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
// distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
// See the GNU General Public License for more details. You should have received a copy of
|
||||
// the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
// Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
//
|
||||
// As a special exception, you may use this file as part of a free software library without
|
||||
// restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
// functions from this file, or you compile this file and link it with other files to produce
|
||||
// an executable, this file does not by itself cause the resulting executable to be covered
|
||||
// by the GNU General Public License. This exception does not however invalidate any other
|
||||
// reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.section .text
|
||||
.align 16
|
||||
// void __TBB_machine_pause( long count );
|
||||
// r32 = count
|
||||
.proc __TBB_machine_pause#
|
||||
.global __TBB_machine_pause#
|
||||
count = r32
|
||||
__TBB_machine_pause:
|
||||
hint.m 0
|
||||
add count=-1,count
|
||||
;;
|
||||
cmp.eq p6,p7=0,count
|
||||
(p7) br.cond.dpnt __TBB_machine_pause
|
||||
(p6) br.ret.sptk.many b0
|
||||
.endp __TBB_machine_pause#
|
||||
@@ -0,0 +1,55 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include <stdint.h>
|
||||
#include <sys/atomic_op.h>
|
||||
|
||||
/* This file must be compiled with gcc. The IBM compiler doesn't seem to
|
||||
support inline assembly statements (October 2007). */
|
||||
|
||||
#ifdef __GNUC__
|
||||
|
||||
int32_t __TBB_machine_cas_32 (volatile void* ptr, int32_t value, int32_t comparand) {
|
||||
__asm__ __volatile__ ("sync\n"); /* memory release operation */
|
||||
compare_and_swap ((atomic_p) ptr, &comparand, value);
|
||||
__asm__ __volatile__ ("isync\n"); /* memory acquire operation */
|
||||
return comparand;
|
||||
}
|
||||
|
||||
int64_t __TBB_machine_cas_64 (volatile void* ptr, int64_t value, int64_t comparand) {
|
||||
__asm__ __volatile__ ("sync\n"); /* memory release operation */
|
||||
compare_and_swaplp ((atomic_l) ptr, &comparand, value);
|
||||
__asm__ __volatile__ ("isync\n"); /* memory acquire operation */
|
||||
return comparand;
|
||||
}
|
||||
|
||||
void __TBB_machine_flush () {
|
||||
__asm__ __volatile__ ("sync\n");
|
||||
}
|
||||
|
||||
void __TBB_machine_lwsync () {
|
||||
__asm__ __volatile__ ("lwsync\n");
|
||||
}
|
||||
|
||||
void __TBB_machine_isync () {
|
||||
__asm__ __volatile__ ("isync\n");
|
||||
}
|
||||
|
||||
#endif /* __GNUC__ */
|
||||
@@ -0,0 +1,72 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
; DO NOT EDIT - AUTOMATICALLY GENERATED FROM .s FILE
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_fetchadd1
|
||||
__TBB_machine_fetchadd1:
|
||||
mov rax,rdx
|
||||
lock xadd [rcx],al
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_fetchstore1
|
||||
__TBB_machine_fetchstore1:
|
||||
mov rax,rdx
|
||||
lock xchg [rcx],al
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_cmpswp1
|
||||
__TBB_machine_cmpswp1:
|
||||
mov rax,r8
|
||||
lock cmpxchg [rcx],dl
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_fetchadd2
|
||||
__TBB_machine_fetchadd2:
|
||||
mov rax,rdx
|
||||
lock xadd [rcx],ax
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_fetchstore2
|
||||
__TBB_machine_fetchstore2:
|
||||
mov rax,rdx
|
||||
lock xchg [rcx],ax
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_cmpswp2
|
||||
__TBB_machine_cmpswp2:
|
||||
mov rax,r8
|
||||
lock cmpxchg [rcx],dx
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_pause
|
||||
__TBB_machine_pause:
|
||||
L1:
|
||||
dw 090f3H; pause
|
||||
add ecx,-1
|
||||
jne L1
|
||||
ret
|
||||
end
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_get_cpu_ctl_env
|
||||
__TBB_get_cpu_ctl_env:
|
||||
stmxcsr [rcx]
|
||||
fstcw [rcx+4]
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_set_cpu_ctl_env
|
||||
__TBB_set_cpu_ctl_env:
|
||||
ldmxcsr [rcx]
|
||||
fldcw [rcx+4]
|
||||
ret
|
||||
end
|
||||
@@ -0,0 +1,76 @@
|
||||
; Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
;
|
||||
; This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
; you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
; version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
; distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
; See the GNU General Public License for more details. You should have received a copy of
|
||||
; the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
; Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
;
|
||||
; As a special exception, you may use this file as part of a free software library without
|
||||
; restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
; functions from this file, or you compile this file and link it with other files to produce
|
||||
; an executable, this file does not by itself cause the resulting executable to be covered
|
||||
; by the GNU General Public License. This exception does not however invalidate any other
|
||||
; reasons why the executable file might be covered by the GNU General Public License.
|
||||
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_try_lock_elided
|
||||
__TBB_machine_try_lock_elided:
|
||||
xor rax, rax
|
||||
mov al, 1
|
||||
BYTE 0F2H
|
||||
xchg al, byte ptr [rcx]
|
||||
xor al, 1
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_unlock_elided
|
||||
__TBB_machine_unlock_elided:
|
||||
BYTE 0F3H
|
||||
mov byte ptr [rcx], 0
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_begin_transaction
|
||||
__TBB_machine_begin_transaction:
|
||||
mov eax, -1
|
||||
BYTE 0C7H
|
||||
BYTE 0F8H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
BYTE 000H
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_end_transaction
|
||||
__TBB_machine_end_transaction:
|
||||
BYTE 00FH
|
||||
BYTE 001H
|
||||
BYTE 0D5H
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_transaction_conflict_abort
|
||||
__TBB_machine_transaction_conflict_abort:
|
||||
BYTE 0C6H
|
||||
BYTE 0F8H
|
||||
BYTE 0FFH ; 12.4.5 Abort argument: lock not free when tested
|
||||
ret
|
||||
.code
|
||||
ALIGN 8
|
||||
PUBLIC __TBB_machine_is_in_transaction
|
||||
__TBB_machine_is_in_transaction:
|
||||
xor eax, eax
|
||||
BYTE 00FH ; _xtest sets or clears ZF
|
||||
BYTE 001H
|
||||
BYTE 0D6H
|
||||
jz rset
|
||||
mov al,1
|
||||
rset:
|
||||
ret
|
||||
end
|
||||
@@ -0,0 +1,180 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__aggregator_impl_H
|
||||
#define __TBB__aggregator_impl_H
|
||||
|
||||
#include "../atomic.h"
|
||||
#if !__TBBMALLOC_BUILD
|
||||
#include "../tbb_profiling.h"
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
namespace interface6 {
|
||||
namespace internal {
|
||||
|
||||
using namespace tbb::internal;
|
||||
|
||||
//! aggregated_operation base class
|
||||
template <typename Derived>
|
||||
class aggregated_operation {
|
||||
public:
|
||||
uintptr_t status;
|
||||
Derived *next;
|
||||
aggregated_operation() : status(0), next(NULL) {}
|
||||
};
|
||||
|
||||
//! Aggregator base class
|
||||
/** An aggregator for collecting operations coming from multiple sources and executing
|
||||
them serially on a single thread. operation_type must be derived from
|
||||
aggregated_operation. The parameter handler_type is a functor that will be passed the
|
||||
list of operations and is expected to handle each operation appropriately, setting the
|
||||
status of each operation to non-zero.*/
|
||||
template < typename operation_type >
|
||||
class aggregator_generic {
|
||||
public:
|
||||
aggregator_generic() : handler_busy(false) { pending_operations = NULL; }
|
||||
|
||||
//! Place operation in list
|
||||
/** Place operation in list and either handle list or wait for operation to
|
||||
complete.
|
||||
long_life_time specifies life time of an operation inserting in an aggregator.
|
||||
"Long" (long_life_time == true) life time operation can be accessed
|
||||
even after executing it.
|
||||
"Short" (long_life_time == false) life time operations can be destroyed
|
||||
during executing so any access to it after executing is invalid.*/
|
||||
template < typename handler_type >
|
||||
void execute(operation_type *op, handler_type &handle_operations, bool long_life_time = true) {
|
||||
operation_type *res;
|
||||
// op->status should be read before inserting the operation in the
|
||||
// aggregator queue since it can become invalid after executing a
|
||||
// handler (if the operation has 'short' life time.)
|
||||
const uintptr_t status = op->status;
|
||||
|
||||
// ITT note: &(op->status) tag is used to cover accesses to this op node. This
|
||||
// thread has created the operation, and now releases it so that the handler
|
||||
// thread may handle the associated operation w/o triggering a race condition;
|
||||
// thus this tag will be acquired just before the operation is handled in the
|
||||
// handle_operations functor.
|
||||
call_itt_notify(releasing, &(op->status));
|
||||
// insert the operation in the queue.
|
||||
do {
|
||||
// ITT may flag the following line as a race; it is a false positive:
|
||||
// This is an atomic read; we don't provide itt_hide_load_word for atomics
|
||||
op->next = res = pending_operations; // NOT A RACE
|
||||
} while (pending_operations.compare_and_swap(op, res) != res);
|
||||
if (!res) { // first in the list; handle the operations.
|
||||
// ITT note: &pending_operations tag covers access to the handler_busy flag,
|
||||
// which this waiting handler thread will try to set before entering
|
||||
// handle_operations.
|
||||
call_itt_notify(acquired, &pending_operations);
|
||||
start_handle_operations(handle_operations);
|
||||
// The operation with 'short' life time can already be destroyed.
|
||||
if (long_life_time)
|
||||
__TBB_ASSERT(op->status, NULL);
|
||||
}
|
||||
// not first; wait for op to be ready.
|
||||
else if (!status) { // operation is blocking here.
|
||||
__TBB_ASSERT(long_life_time, "The blocking operation cannot have 'short' life time. Since it can already be destroyed.");
|
||||
call_itt_notify(prepare, &(op->status));
|
||||
spin_wait_while_eq(op->status, uintptr_t(0));
|
||||
itt_load_word_with_acquire(op->status);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
//! An atomically updated list (aka mailbox) of pending operations
|
||||
atomic<operation_type *> pending_operations;
|
||||
//! Controls thread access to handle_operations
|
||||
uintptr_t handler_busy;
|
||||
|
||||
//! Trigger the handling of operations when the handler is free
|
||||
template < typename handler_type >
|
||||
void start_handle_operations( handler_type &handle_operations ) {
|
||||
operation_type *op_list;
|
||||
|
||||
// ITT note: &handler_busy tag covers access to pending_operations as it is passed
|
||||
// between active and waiting handlers. Below, the waiting handler waits until
|
||||
// the active handler releases, and the waiting handler acquires &handler_busy as
|
||||
// it becomes the active_handler. The release point is at the end of this
|
||||
// function, when all operations in pending_operations have been handled by the
|
||||
// owner of this aggregator.
|
||||
call_itt_notify(prepare, &handler_busy);
|
||||
// get the handler_busy:
|
||||
// only one thread can possibly spin here at a time
|
||||
spin_wait_until_eq(handler_busy, uintptr_t(0));
|
||||
call_itt_notify(acquired, &handler_busy);
|
||||
// acquire fence not necessary here due to causality rule and surrounding atomics
|
||||
__TBB_store_with_release(handler_busy, uintptr_t(1));
|
||||
|
||||
// ITT note: &pending_operations tag covers access to the handler_busy flag
|
||||
// itself. Capturing the state of the pending_operations signifies that
|
||||
// handler_busy has been set and a new active handler will now process that list's
|
||||
// operations.
|
||||
call_itt_notify(releasing, &pending_operations);
|
||||
// grab pending_operations
|
||||
op_list = pending_operations.fetch_and_store(NULL);
|
||||
|
||||
// handle all the operations
|
||||
handle_operations(op_list);
|
||||
|
||||
// release the handler
|
||||
itt_store_word_with_release(handler_busy, uintptr_t(0));
|
||||
}
|
||||
};
|
||||
|
||||
template < typename handler_type, typename operation_type >
|
||||
class aggregator : public aggregator_generic<operation_type> {
|
||||
handler_type handle_operations;
|
||||
public:
|
||||
aggregator() {}
|
||||
explicit aggregator(handler_type h) : handle_operations(h) {}
|
||||
|
||||
void initialize_handler(handler_type h) { handle_operations = h; }
|
||||
|
||||
void execute(operation_type *op) {
|
||||
aggregator_generic<operation_type>::execute(op, handle_operations);
|
||||
}
|
||||
};
|
||||
|
||||
// the most-compatible friend declaration (vs, gcc, icc) is
|
||||
// template<class U, class V> friend class aggregating_functor;
|
||||
template<typename aggregating_class, typename operation_list>
|
||||
class aggregating_functor {
|
||||
aggregating_class *fi;
|
||||
public:
|
||||
aggregating_functor() {}
|
||||
aggregating_functor(aggregating_class *fi_) : fi(fi_) {}
|
||||
void operator()(operation_list* op_list) { fi->handle_operations(op_list); }
|
||||
};
|
||||
|
||||
} // namespace internal
|
||||
} // namespace interface6
|
||||
|
||||
namespace internal {
|
||||
using interface6::internal::aggregated_operation;
|
||||
using interface6::internal::aggregator_generic;
|
||||
using interface6::internal::aggregator;
|
||||
using interface6::internal::aggregating_functor;
|
||||
} // namespace internal
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
#endif // __TBB__aggregator_impl_H
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,757 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__flow_graph_impl_H
|
||||
#define __TBB__flow_graph_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
namespace internal {
|
||||
|
||||
namespace graph_policy_namespace {
|
||||
enum graph_buffer_policy { rejecting, reserving, queueing, tag_matching };
|
||||
}
|
||||
|
||||
// -------------- function_body containers ----------------------
|
||||
|
||||
//! A functor that takes no input and generates a value of type Output
|
||||
template< typename Output >
|
||||
class source_body : tbb::internal::no_assign {
|
||||
public:
|
||||
virtual ~source_body() {}
|
||||
virtual bool operator()(Output &output) = 0;
|
||||
virtual source_body* clone() = 0;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
virtual void reset_body() = 0;
|
||||
#endif
|
||||
};
|
||||
|
||||
//! The leaf for source_body
|
||||
template< typename Output, typename Body>
|
||||
class source_body_leaf : public source_body<Output> {
|
||||
public:
|
||||
source_body_leaf( const Body &_body ) : body(_body), init_body(_body) { }
|
||||
/*override*/ bool operator()(Output &output) { return body( output ); }
|
||||
/*override*/ source_body_leaf* clone() {
|
||||
return new source_body_leaf< Output, Body >(init_body);
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
Body get_body() { return body; }
|
||||
private:
|
||||
Body body;
|
||||
Body init_body;
|
||||
};
|
||||
|
||||
//! A functor that takes an Input and generates an Output
|
||||
template< typename Input, typename Output >
|
||||
class function_body : tbb::internal::no_assign {
|
||||
public:
|
||||
virtual ~function_body() {}
|
||||
virtual Output operator()(const Input &input) = 0;
|
||||
virtual function_body* clone() = 0;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
virtual void reset_body() = 0;
|
||||
#endif
|
||||
};
|
||||
|
||||
//! the leaf for function_body
|
||||
template <typename Input, typename Output, typename B>
|
||||
class function_body_leaf : public function_body< Input, Output > {
|
||||
public:
|
||||
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
|
||||
Output operator()(const Input &i) { return body(i); }
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
B get_body() { return body; }
|
||||
/*override*/ function_body_leaf* clone() {
|
||||
return new function_body_leaf< Input, Output, B >(init_body);
|
||||
}
|
||||
private:
|
||||
B body;
|
||||
B init_body;
|
||||
};
|
||||
|
||||
//! the leaf for function_body specialized for Input and output of continue_msg
|
||||
template <typename B>
|
||||
class function_body_leaf< continue_msg, continue_msg, B> : public function_body< continue_msg, continue_msg > {
|
||||
public:
|
||||
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
|
||||
continue_msg operator()( const continue_msg &i ) {
|
||||
body(i);
|
||||
return i;
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
B get_body() { return body; }
|
||||
/*override*/ function_body_leaf* clone() {
|
||||
return new function_body_leaf< continue_msg, continue_msg, B >(init_body);
|
||||
}
|
||||
private:
|
||||
B body;
|
||||
B init_body;
|
||||
};
|
||||
|
||||
//! the leaf for function_body specialized for Output of continue_msg
|
||||
template <typename Input, typename B>
|
||||
class function_body_leaf< Input, continue_msg, B> : public function_body< Input, continue_msg > {
|
||||
public:
|
||||
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
|
||||
continue_msg operator()(const Input &i) {
|
||||
body(i);
|
||||
return continue_msg();
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
B get_body() { return body; }
|
||||
/*override*/ function_body_leaf* clone() {
|
||||
return new function_body_leaf< Input, continue_msg, B >(init_body);
|
||||
}
|
||||
private:
|
||||
B body;
|
||||
B init_body;
|
||||
};
|
||||
|
||||
//! the leaf for function_body specialized for Input of continue_msg
|
||||
template <typename Output, typename B>
|
||||
class function_body_leaf< continue_msg, Output, B > : public function_body< continue_msg, Output > {
|
||||
public:
|
||||
function_body_leaf( const B &_body ) : body(_body), init_body(_body) { }
|
||||
Output operator()(const continue_msg &i) {
|
||||
return body(i);
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
B get_body() { return body; }
|
||||
/*override*/ function_body_leaf* clone() {
|
||||
return new function_body_leaf< continue_msg, Output, B >(init_body);
|
||||
}
|
||||
private:
|
||||
B body;
|
||||
B init_body;
|
||||
};
|
||||
|
||||
//! function_body that takes an Input and a set of output ports
|
||||
template<typename Input, typename OutputSet>
|
||||
class multifunction_body : tbb::internal::no_assign {
|
||||
public:
|
||||
virtual ~multifunction_body () {}
|
||||
virtual void operator()(const Input &/* input*/, OutputSet &/*oset*/) = 0;
|
||||
virtual multifunction_body* clone() = 0;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
virtual void reset_body() = 0;
|
||||
#endif
|
||||
};
|
||||
|
||||
//! leaf for multifunction. OutputSet can be a std::tuple or a vector.
|
||||
template<typename Input, typename OutputSet, typename B>
|
||||
class multifunction_body_leaf : public multifunction_body<Input, OutputSet> {
|
||||
public:
|
||||
multifunction_body_leaf(const B &_body) : body(_body), init_body(_body) { }
|
||||
void operator()(const Input &input, OutputSet &oset) {
|
||||
body(input, oset); // body may explicitly put() to one or more of oset.
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void reset_body() {
|
||||
body = init_body;
|
||||
}
|
||||
#endif
|
||||
B get_body() { return body; }
|
||||
/*override*/ multifunction_body_leaf* clone() {
|
||||
return new multifunction_body_leaf<Input, OutputSet,B>(init_body);
|
||||
}
|
||||
private:
|
||||
B body;
|
||||
B init_body;
|
||||
};
|
||||
|
||||
// --------------------------- end of function_body containers ------------------------
|
||||
|
||||
// --------------------------- node task bodies ---------------------------------------
|
||||
|
||||
//! A task that calls a node's forward_task function
|
||||
template< typename NodeType >
|
||||
class forward_task_bypass : public task {
|
||||
|
||||
NodeType &my_node;
|
||||
|
||||
public:
|
||||
|
||||
forward_task_bypass( NodeType &n ) : my_node(n) {}
|
||||
|
||||
task *execute() {
|
||||
task * new_task = my_node.forward_task();
|
||||
if (new_task == SUCCESSFULLY_ENQUEUED) new_task = NULL;
|
||||
return new_task;
|
||||
}
|
||||
};
|
||||
|
||||
//! A task that calls a node's apply_body_bypass function, passing in an input of type Input
|
||||
// return the task* unless it is SUCCESSFULLY_ENQUEUED, in which case return NULL
|
||||
template< typename NodeType, typename Input >
|
||||
class apply_body_task_bypass : public task {
|
||||
|
||||
NodeType &my_node;
|
||||
Input my_input;
|
||||
|
||||
public:
|
||||
|
||||
apply_body_task_bypass( NodeType &n, const Input &i ) : my_node(n), my_input(i) {}
|
||||
|
||||
task *execute() {
|
||||
task * next_task = my_node.apply_body_bypass( my_input );
|
||||
if(next_task == SUCCESSFULLY_ENQUEUED) next_task = NULL;
|
||||
return next_task;
|
||||
}
|
||||
};
|
||||
|
||||
//! A task that calls a node's apply_body function with no input
|
||||
template< typename NodeType >
|
||||
class source_task_bypass : public task {
|
||||
|
||||
NodeType &my_node;
|
||||
|
||||
public:
|
||||
|
||||
source_task_bypass( NodeType &n ) : my_node(n) {}
|
||||
|
||||
task *execute() {
|
||||
task *new_task = my_node.apply_body_bypass( );
|
||||
if(new_task == SUCCESSFULLY_ENQUEUED) return NULL;
|
||||
return new_task;
|
||||
}
|
||||
};
|
||||
|
||||
// ------------------------ end of node task bodies -----------------------------------
|
||||
|
||||
//! An empty functor that takes an Input and returns a default constructed Output
|
||||
template< typename Input, typename Output >
|
||||
struct empty_body {
|
||||
Output operator()( const Input & ) const { return Output(); }
|
||||
};
|
||||
|
||||
//! A node_cache maintains a std::queue of elements of type T. Each operation is protected by a lock.
|
||||
template< typename T, typename M=spin_mutex >
|
||||
class node_cache {
|
||||
public:
|
||||
|
||||
typedef size_t size_type;
|
||||
|
||||
bool empty() {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
return internal_empty();
|
||||
}
|
||||
|
||||
void add( T &n ) {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
internal_push(n);
|
||||
}
|
||||
|
||||
void remove( T &n ) {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
for ( size_t i = internal_size(); i != 0; --i ) {
|
||||
T &s = internal_pop();
|
||||
if ( &s == &n ) return; // only remove one predecessor per request
|
||||
internal_push(s);
|
||||
}
|
||||
}
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<T *> predecessor_vector_type;
|
||||
void internal_add_built_predecessor( T &n ) {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
my_built_predecessors.add_edge(n);
|
||||
}
|
||||
|
||||
void internal_delete_built_predecessor( T &n ) {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
my_built_predecessors.delete_edge(n);
|
||||
}
|
||||
|
||||
void copy_predecessors( predecessor_vector_type &v) {
|
||||
typename my_mutex_type::scoped_lock lock( my_mutex );
|
||||
my_built_predecessors.copy_edges(v);
|
||||
}
|
||||
|
||||
size_t predecessor_count() {
|
||||
typename my_mutex_type::scoped_lock lock(my_mutex);
|
||||
return (size_t)(my_built_predecessors.edge_count());
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
|
||||
protected:
|
||||
|
||||
typedef M my_mutex_type;
|
||||
my_mutex_type my_mutex;
|
||||
std::queue< T * > my_q;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
edge_container<T> my_built_predecessors;
|
||||
#endif
|
||||
|
||||
// Assumes lock is held
|
||||
inline bool internal_empty( ) {
|
||||
return my_q.empty();
|
||||
}
|
||||
|
||||
// Assumes lock is held
|
||||
inline size_type internal_size( ) {
|
||||
return my_q.size();
|
||||
}
|
||||
|
||||
// Assumes lock is held
|
||||
inline void internal_push( T &n ) {
|
||||
my_q.push(&n);
|
||||
}
|
||||
|
||||
// Assumes lock is held
|
||||
inline T &internal_pop() {
|
||||
T *v = my_q.front();
|
||||
my_q.pop();
|
||||
return *v;
|
||||
}
|
||||
|
||||
};
|
||||
|
||||
//! A cache of predecessors that only supports try_get
|
||||
template< typename T, typename M=spin_mutex >
|
||||
class predecessor_cache : public node_cache< sender<T>, M > {
|
||||
public:
|
||||
typedef M my_mutex_type;
|
||||
typedef T output_type;
|
||||
typedef sender<output_type> predecessor_type;
|
||||
typedef receiver<output_type> successor_type;
|
||||
|
||||
predecessor_cache( ) : my_owner( NULL ) { }
|
||||
|
||||
void set_owner( successor_type *owner ) { my_owner = owner; }
|
||||
|
||||
bool get_item( output_type &v ) {
|
||||
|
||||
bool msg = false;
|
||||
|
||||
do {
|
||||
predecessor_type *src;
|
||||
{
|
||||
typename my_mutex_type::scoped_lock lock(this->my_mutex);
|
||||
if ( this->internal_empty() ) {
|
||||
break;
|
||||
}
|
||||
src = &this->internal_pop();
|
||||
}
|
||||
|
||||
// Try to get from this sender
|
||||
msg = src->try_get( v );
|
||||
|
||||
if (msg == false) {
|
||||
// Relinquish ownership of the edge
|
||||
if ( my_owner)
|
||||
src->register_successor( *my_owner );
|
||||
} else {
|
||||
// Retain ownership of the edge
|
||||
this->add(*src);
|
||||
}
|
||||
} while ( msg == false );
|
||||
return msg;
|
||||
}
|
||||
|
||||
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
if(my_owner) {
|
||||
for(;;) {
|
||||
predecessor_type *src;
|
||||
{
|
||||
if(this->internal_empty()) break;
|
||||
src = &this->internal_pop();
|
||||
}
|
||||
src->register_successor( *my_owner);
|
||||
}
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
if (f&rf_extract && my_owner)
|
||||
my_built_predecessors.receiver_extract(*my_owner);
|
||||
__TBB_ASSERT(!(f&rf_extract) || this->internal_empty(), "predecessor cache not empty");
|
||||
#endif
|
||||
}
|
||||
|
||||
protected:
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
using node_cache< sender<T>, M >::my_built_predecessors;
|
||||
#endif
|
||||
successor_type *my_owner;
|
||||
};
|
||||
|
||||
//! An cache of predecessors that supports requests and reservations
|
||||
template< typename T, typename M=spin_mutex >
|
||||
class reservable_predecessor_cache : public predecessor_cache< T, M > {
|
||||
public:
|
||||
typedef M my_mutex_type;
|
||||
typedef T output_type;
|
||||
typedef sender<T> predecessor_type;
|
||||
typedef receiver<T> successor_type;
|
||||
|
||||
reservable_predecessor_cache( ) : reserved_src(NULL) { }
|
||||
|
||||
bool
|
||||
try_reserve( output_type &v ) {
|
||||
bool msg = false;
|
||||
|
||||
do {
|
||||
{
|
||||
typename my_mutex_type::scoped_lock lock(this->my_mutex);
|
||||
if ( reserved_src || this->internal_empty() )
|
||||
return false;
|
||||
|
||||
reserved_src = &this->internal_pop();
|
||||
}
|
||||
|
||||
// Try to get from this sender
|
||||
msg = reserved_src->try_reserve( v );
|
||||
|
||||
if (msg == false) {
|
||||
typename my_mutex_type::scoped_lock lock(this->my_mutex);
|
||||
// Relinquish ownership of the edge
|
||||
reserved_src->register_successor( *this->my_owner );
|
||||
reserved_src = NULL;
|
||||
} else {
|
||||
// Retain ownership of the edge
|
||||
this->add( *reserved_src );
|
||||
}
|
||||
} while ( msg == false );
|
||||
|
||||
return msg;
|
||||
}
|
||||
|
||||
bool
|
||||
try_release( ) {
|
||||
reserved_src->try_release( );
|
||||
reserved_src = NULL;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool
|
||||
try_consume( ) {
|
||||
reserved_src->try_consume( );
|
||||
reserved_src = NULL;
|
||||
return true;
|
||||
}
|
||||
|
||||
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
reserved_src = NULL;
|
||||
predecessor_cache<T,M>::reset(__TBB_PFG_RESET_ARG(f));
|
||||
}
|
||||
|
||||
private:
|
||||
predecessor_type *reserved_src;
|
||||
};
|
||||
|
||||
|
||||
//! An abstract cache of successors
|
||||
template<typename T, typename M=spin_rw_mutex >
|
||||
class successor_cache : tbb::internal::no_copy {
|
||||
protected:
|
||||
|
||||
typedef M my_mutex_type;
|
||||
my_mutex_type my_mutex;
|
||||
|
||||
typedef receiver<T> *pointer_type;
|
||||
typedef std::list< pointer_type > my_successors_type;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
edge_container<receiver<T> > my_built_successors;
|
||||
#endif
|
||||
my_successors_type my_successors;
|
||||
|
||||
sender<T> *my_owner;
|
||||
|
||||
public:
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<pointer_type> successor_vector_type;
|
||||
void internal_add_built_successor( receiver<T> &r) {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_built_successors.add_edge( r );
|
||||
}
|
||||
|
||||
void internal_delete_built_successor( receiver<T> &r) {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_built_successors.delete_edge(r);
|
||||
}
|
||||
|
||||
void copy_successors( successor_vector_type &v) {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, false);
|
||||
my_built_successors.copy_edges(v);
|
||||
}
|
||||
|
||||
size_t successor_count() {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex,false);
|
||||
return my_built_successors.edge_count();
|
||||
}
|
||||
|
||||
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
if (f&rf_extract && my_owner)
|
||||
my_built_successors.sender_extract(*my_owner);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
|
||||
successor_cache( ) : my_owner(NULL) {}
|
||||
|
||||
void set_owner( sender<T> *owner ) { my_owner = owner; }
|
||||
|
||||
virtual ~successor_cache() {}
|
||||
|
||||
void register_successor( receiver<T> &r ) {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_successors.push_back( &r );
|
||||
}
|
||||
|
||||
void remove_successor( receiver<T> &r ) {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
for ( typename my_successors_type::iterator i = my_successors.begin();
|
||||
i != my_successors.end(); ++i ) {
|
||||
if ( *i == & r ) {
|
||||
my_successors.erase(i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool empty() {
|
||||
typename my_mutex_type::scoped_lock l(my_mutex, false);
|
||||
return my_successors.empty();
|
||||
}
|
||||
|
||||
void clear() {
|
||||
my_successors.clear();
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
my_built_successors.clear();
|
||||
#endif
|
||||
}
|
||||
|
||||
virtual task * try_put_task( const T &t ) = 0;
|
||||
};
|
||||
|
||||
//! An abstract cache of successors, specialized to continue_msg
|
||||
template<>
|
||||
class successor_cache< continue_msg > : tbb::internal::no_copy {
|
||||
protected:
|
||||
|
||||
typedef spin_rw_mutex my_mutex_type;
|
||||
my_mutex_type my_mutex;
|
||||
|
||||
typedef receiver<continue_msg> *pointer_type;
|
||||
typedef std::list< pointer_type > my_successors_type;
|
||||
my_successors_type my_successors;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
edge_container<receiver<continue_msg> > my_built_successors;
|
||||
#endif
|
||||
|
||||
sender<continue_msg> *my_owner;
|
||||
|
||||
public:
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<pointer_type> successor_vector_type;
|
||||
void internal_add_built_successor( receiver<continue_msg> &r) {
|
||||
my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_built_successors.add_edge( r );
|
||||
}
|
||||
|
||||
void internal_delete_built_successor( receiver<continue_msg> &r) {
|
||||
my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_built_successors.delete_edge(r);
|
||||
}
|
||||
|
||||
void copy_successors( successor_vector_type &v) {
|
||||
my_mutex_type::scoped_lock l(my_mutex, false);
|
||||
my_built_successors.copy_edges(v);
|
||||
}
|
||||
|
||||
size_t successor_count() {
|
||||
my_mutex_type::scoped_lock l(my_mutex,false);
|
||||
return my_built_successors.edge_count();
|
||||
}
|
||||
|
||||
void reset( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
if (f&rf_extract && my_owner)
|
||||
my_built_successors.sender_extract(*my_owner);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
|
||||
successor_cache( ) : my_owner(NULL) {}
|
||||
|
||||
void set_owner( sender<continue_msg> *owner ) { my_owner = owner; }
|
||||
|
||||
virtual ~successor_cache() {}
|
||||
|
||||
void register_successor( receiver<continue_msg> &r ) {
|
||||
my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
my_successors.push_back( &r );
|
||||
if ( my_owner && r.is_continue_receiver() ) {
|
||||
r.register_predecessor( *my_owner );
|
||||
}
|
||||
}
|
||||
|
||||
void remove_successor( receiver<continue_msg> &r ) {
|
||||
my_mutex_type::scoped_lock l(my_mutex, true);
|
||||
for ( my_successors_type::iterator i = my_successors.begin();
|
||||
i != my_successors.end(); ++i ) {
|
||||
if ( *i == & r ) {
|
||||
// TODO: Check if we need to test for continue_receiver before
|
||||
// removing from r.
|
||||
if ( my_owner )
|
||||
r.remove_predecessor( *my_owner );
|
||||
my_successors.erase(i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool empty() {
|
||||
my_mutex_type::scoped_lock l(my_mutex, false);
|
||||
return my_successors.empty();
|
||||
}
|
||||
|
||||
void clear() {
|
||||
my_successors.clear();
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
my_built_successors.clear();
|
||||
#endif
|
||||
}
|
||||
|
||||
virtual task * try_put_task( const continue_msg &t ) = 0;
|
||||
|
||||
};
|
||||
|
||||
//! A cache of successors that are broadcast to
|
||||
template<typename T, typename M=spin_rw_mutex>
|
||||
class broadcast_cache : public successor_cache<T, M> {
|
||||
typedef M my_mutex_type;
|
||||
typedef std::list< receiver<T> * > my_successors_type;
|
||||
|
||||
public:
|
||||
|
||||
broadcast_cache( ) {}
|
||||
|
||||
// as above, but call try_put_task instead, and return the last task we received (if any)
|
||||
/*override*/ task * try_put_task( const T &t ) {
|
||||
task * last_task = NULL;
|
||||
bool upgraded = true;
|
||||
typename my_mutex_type::scoped_lock l(this->my_mutex, upgraded);
|
||||
typename my_successors_type::iterator i = this->my_successors.begin();
|
||||
while ( i != this->my_successors.end() ) {
|
||||
task *new_task = (*i)->try_put_task(t);
|
||||
last_task = combine_tasks(last_task, new_task); // enqueue if necessary
|
||||
if(new_task) {
|
||||
++i;
|
||||
}
|
||||
else { // failed
|
||||
if ( (*i)->register_predecessor(*this->my_owner) ) {
|
||||
if (!upgraded) {
|
||||
l.upgrade_to_writer();
|
||||
upgraded = true;
|
||||
}
|
||||
i = this->my_successors.erase(i);
|
||||
} else {
|
||||
++i;
|
||||
}
|
||||
}
|
||||
}
|
||||
return last_task;
|
||||
}
|
||||
|
||||
};
|
||||
|
||||
//! A cache of successors that are put in a round-robin fashion
|
||||
template<typename T, typename M=spin_rw_mutex >
|
||||
class round_robin_cache : public successor_cache<T, M> {
|
||||
typedef size_t size_type;
|
||||
typedef M my_mutex_type;
|
||||
typedef std::list< receiver<T> * > my_successors_type;
|
||||
|
||||
public:
|
||||
|
||||
round_robin_cache( ) {}
|
||||
|
||||
size_type size() {
|
||||
typename my_mutex_type::scoped_lock l(this->my_mutex, false);
|
||||
return this->my_successors.size();
|
||||
}
|
||||
|
||||
/*override*/task *try_put_task( const T &t ) {
|
||||
bool upgraded = true;
|
||||
typename my_mutex_type::scoped_lock l(this->my_mutex, upgraded);
|
||||
typename my_successors_type::iterator i = this->my_successors.begin();
|
||||
while ( i != this->my_successors.end() ) {
|
||||
task *new_task = (*i)->try_put_task(t);
|
||||
if ( new_task ) {
|
||||
return new_task;
|
||||
} else {
|
||||
if ( (*i)->register_predecessor(*this->my_owner) ) {
|
||||
if (!upgraded) {
|
||||
l.upgrade_to_writer();
|
||||
upgraded = true;
|
||||
}
|
||||
i = this->my_successors.erase(i);
|
||||
}
|
||||
else {
|
||||
++i;
|
||||
}
|
||||
}
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
class decrementer : public continue_receiver, tbb::internal::no_copy {
|
||||
|
||||
T *my_node;
|
||||
|
||||
task *execute() {
|
||||
return my_node->decrement_counter();
|
||||
}
|
||||
|
||||
public:
|
||||
|
||||
typedef continue_msg input_type;
|
||||
typedef continue_msg output_type;
|
||||
decrementer( int number_of_predecessors = 0 ) : continue_receiver( number_of_predecessors ) { }
|
||||
void set_owner( T *node ) { my_node = node; }
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
#endif // __TBB__flow_graph_impl_H
|
||||
|
||||
@@ -0,0 +1,453 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__flow_graph_indexer_impl_H
|
||||
#define __TBB__flow_graph_indexer_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#include "tbb/internal/_flow_graph_types_impl.h"
|
||||
|
||||
namespace internal {
|
||||
|
||||
// Output of the indexer_node is a tbb::flow::tagged_msg, and will be of
|
||||
// the form tagged_msg<tag, result>
|
||||
// where the value of tag will indicate which result was put to the
|
||||
// successor.
|
||||
|
||||
template<typename IndexerNodeBaseType, typename T, size_t K>
|
||||
task* do_try_put(const T &v, void *p) {
|
||||
typename IndexerNodeBaseType::output_type o(K, v);
|
||||
return reinterpret_cast<IndexerNodeBaseType *>(p)->try_put_task(&o);
|
||||
}
|
||||
|
||||
template<typename TupleTypes,int N>
|
||||
struct indexer_helper {
|
||||
template<typename IndexerNodeBaseType, typename PortTuple>
|
||||
static inline void set_indexer_node_pointer(PortTuple &my_input, IndexerNodeBaseType *p) {
|
||||
typedef typename tuple_element<N-1, TupleTypes>::type T;
|
||||
task *(*indexer_node_put_task)(const T&, void *) = do_try_put<IndexerNodeBaseType, T, N-1>;
|
||||
tbb::flow::get<N-1>(my_input).set_up(p, indexer_node_put_task);
|
||||
indexer_helper<TupleTypes,N-1>::template set_indexer_node_pointer<IndexerNodeBaseType,PortTuple>(my_input, p);
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
template<typename InputTuple>
|
||||
static inline void reset_inputs(InputTuple &my_input, reset_flags f) {
|
||||
join_helper<N-1>::reset_inputs(my_input, f);
|
||||
tbb::flow::get<N-1>(my_input).reset_receiver(f);
|
||||
}
|
||||
#endif
|
||||
};
|
||||
|
||||
template<typename TupleTypes>
|
||||
struct indexer_helper<TupleTypes,1> {
|
||||
template<typename IndexerNodeBaseType, typename PortTuple>
|
||||
static inline void set_indexer_node_pointer(PortTuple &my_input, IndexerNodeBaseType *p) {
|
||||
typedef typename tuple_element<0, TupleTypes>::type T;
|
||||
task *(*indexer_node_put_task)(const T&, void *) = do_try_put<IndexerNodeBaseType, T, 0>;
|
||||
tbb::flow::get<0>(my_input).set_up(p, indexer_node_put_task);
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
template<typename InputTuple>
|
||||
static inline void reset_inputs(InputTuple &my_input, reset_flags f) {
|
||||
tbb::flow::get<0>(my_input).reset_receiver(f);
|
||||
}
|
||||
#endif
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
class indexer_input_port : public receiver<T> {
|
||||
private:
|
||||
void* my_indexer_ptr;
|
||||
typedef task* (* forward_function_ptr)(T const &, void* );
|
||||
forward_function_ptr my_try_put_task;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
spin_mutex my_pred_mutex;
|
||||
edge_container<sender<T> > my_built_predecessors;
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
public:
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
indexer_input_port() : my_pred_mutex() {}
|
||||
indexer_input_port( const indexer_input_port & /*other*/ ) : receiver<T>(), my_pred_mutex() {
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
void set_up(void *p, forward_function_ptr f) {
|
||||
my_indexer_ptr = p;
|
||||
my_try_put_task = f;
|
||||
}
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<sender<T> *> predecessor_vector_type;
|
||||
/*override*/size_t predecessor_count() {
|
||||
spin_mutex::scoped_lock l(my_pred_mutex);
|
||||
return my_built_predecessors.edge_count();
|
||||
}
|
||||
/*override*/void internal_add_built_predecessor(sender<T> &p) {
|
||||
spin_mutex::scoped_lock l(my_pred_mutex);
|
||||
my_built_predecessors.add_edge(p);
|
||||
}
|
||||
/*override*/void internal_delete_built_predecessor(sender<T> &p) {
|
||||
spin_mutex::scoped_lock l(my_pred_mutex);
|
||||
my_built_predecessors.delete_edge(p);
|
||||
}
|
||||
/*override*/void copy_predecessors( predecessor_vector_type &v) {
|
||||
spin_mutex::scoped_lock l(my_pred_mutex);
|
||||
return my_built_predecessors.copy_edges(v);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
protected:
|
||||
template< typename R, typename B > friend class run_and_put_task;
|
||||
template<typename X, typename Y> friend class internal::broadcast_cache;
|
||||
template<typename X, typename Y> friend class internal::round_robin_cache;
|
||||
task *try_put_task(const T &v) {
|
||||
return my_try_put_task(v, my_indexer_ptr);
|
||||
}
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
public:
|
||||
/*override*/void reset_receiver(__TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
if(f&rf_extract) my_built_predecessors.receiver_extract(*this);
|
||||
}
|
||||
#else
|
||||
/*override*/void reset_receiver(__TBB_PFG_RESET_ARG(reset_flags /*f*/)) { }
|
||||
#endif
|
||||
|
||||
};
|
||||
|
||||
template<typename InputTuple, typename OutputType, typename StructTypes>
|
||||
class indexer_node_FE {
|
||||
public:
|
||||
static const int N = tbb::flow::tuple_size<InputTuple>::value;
|
||||
typedef OutputType output_type;
|
||||
typedef InputTuple input_type;
|
||||
|
||||
input_type &input_ports() { return my_inputs; }
|
||||
protected:
|
||||
input_type my_inputs;
|
||||
};
|
||||
|
||||
//! indexer_node_base
|
||||
template<typename InputTuple, typename OutputType, typename StructTypes>
|
||||
class indexer_node_base : public graph_node, public indexer_node_FE<InputTuple, OutputType,StructTypes>,
|
||||
public sender<OutputType> {
|
||||
protected:
|
||||
using graph_node::my_graph;
|
||||
public:
|
||||
static const size_t N = tbb::flow::tuple_size<InputTuple>::value;
|
||||
typedef OutputType output_type;
|
||||
typedef StructTypes tuple_types;
|
||||
typedef receiver<output_type> successor_type;
|
||||
typedef indexer_node_FE<InputTuple, output_type,StructTypes> input_ports_type;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<successor_type *> successor_vector_type;
|
||||
#endif
|
||||
|
||||
private:
|
||||
// ----------- Aggregator ------------
|
||||
enum op_type { reg_succ, rem_succ, try__put_task
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
, add_blt_succ, del_blt_succ,
|
||||
blt_succ_cnt, blt_succ_cpy
|
||||
#endif
|
||||
};
|
||||
enum op_stat {WAIT=0, SUCCEEDED, FAILED};
|
||||
typedef indexer_node_base<InputTuple,output_type,StructTypes> my_class;
|
||||
|
||||
class indexer_node_base_operation : public aggregated_operation<indexer_node_base_operation> {
|
||||
public:
|
||||
char type;
|
||||
union {
|
||||
output_type const *my_arg;
|
||||
successor_type *my_succ;
|
||||
task *bypass_t;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
size_t cnt_val;
|
||||
successor_vector_type *succv;
|
||||
#endif
|
||||
};
|
||||
indexer_node_base_operation(const output_type* e, op_type t) :
|
||||
type(char(t)), my_arg(e) {}
|
||||
indexer_node_base_operation(const successor_type &s, op_type t) : type(char(t)),
|
||||
my_succ(const_cast<successor_type *>(&s)) {}
|
||||
indexer_node_base_operation(op_type t) : type(char(t)) {}
|
||||
};
|
||||
|
||||
typedef internal::aggregating_functor<my_class, indexer_node_base_operation> my_handler;
|
||||
friend class internal::aggregating_functor<my_class, indexer_node_base_operation>;
|
||||
aggregator<my_handler, indexer_node_base_operation> my_aggregator;
|
||||
|
||||
void handle_operations(indexer_node_base_operation* op_list) {
|
||||
indexer_node_base_operation *current;
|
||||
while(op_list) {
|
||||
current = op_list;
|
||||
op_list = op_list->next;
|
||||
switch(current->type) {
|
||||
|
||||
case reg_succ:
|
||||
my_successors.register_successor(*(current->my_succ));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
|
||||
case rem_succ:
|
||||
my_successors.remove_successor(*(current->my_succ));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
case try__put_task: {
|
||||
current->bypass_t = my_successors.try_put_task(*(current->my_arg));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED); // return of try_put_task actual return value
|
||||
}
|
||||
break;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
case add_blt_succ:
|
||||
my_successors.internal_add_built_successor(*(current->my_succ));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
case del_blt_succ:
|
||||
my_successors.internal_delete_built_successor(*(current->my_succ));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
case blt_succ_cnt:
|
||||
current->cnt_val = my_successors.successor_count();
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
case blt_succ_cpy:
|
||||
my_successors.copy_successors(*(current->succv));
|
||||
__TBB_store_with_release(current->status, SUCCEEDED);
|
||||
break;
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
}
|
||||
}
|
||||
}
|
||||
// ---------- end aggregator -----------
|
||||
public:
|
||||
indexer_node_base(graph& g) : graph_node(g), input_ports_type() {
|
||||
indexer_helper<StructTypes,N>::set_indexer_node_pointer(this->my_inputs, this);
|
||||
my_successors.set_owner(this);
|
||||
my_aggregator.initialize_handler(my_handler(this));
|
||||
}
|
||||
|
||||
indexer_node_base(const indexer_node_base& other) : graph_node(other.my_graph), input_ports_type(), sender<output_type>() {
|
||||
indexer_helper<StructTypes,N>::set_indexer_node_pointer(this->my_inputs, this);
|
||||
my_successors.set_owner(this);
|
||||
my_aggregator.initialize_handler(my_handler(this));
|
||||
}
|
||||
|
||||
bool register_successor(successor_type &r) {
|
||||
indexer_node_base_operation op_data(r, reg_succ);
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.status == SUCCEEDED;
|
||||
}
|
||||
|
||||
bool remove_successor( successor_type &r) {
|
||||
indexer_node_base_operation op_data(r, rem_succ);
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.status == SUCCEEDED;
|
||||
}
|
||||
|
||||
task * try_put_task(output_type const *v) {
|
||||
indexer_node_base_operation op_data(v, try__put_task);
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.bypass_t;
|
||||
}
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
void internal_add_built_successor( successor_type &r) {
|
||||
indexer_node_base_operation op_data(r, add_blt_succ);
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
|
||||
void internal_delete_built_successor( successor_type &r) {
|
||||
indexer_node_base_operation op_data(r, del_blt_succ);
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
|
||||
size_t successor_count() {
|
||||
indexer_node_base_operation op_data(blt_succ_cnt);
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.cnt_val;
|
||||
}
|
||||
|
||||
void copy_successors( successor_vector_type &v) {
|
||||
indexer_node_base_operation op_data(blt_succ_cpy);
|
||||
op_data.succv = &v;
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
protected:
|
||||
/*override*/void reset(__TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
my_successors.reset(f);
|
||||
indexer_helper<StructTypes,N>::reset_inputs(this->my_inputs, f);
|
||||
#endif
|
||||
}
|
||||
|
||||
private:
|
||||
broadcast_cache<output_type, null_rw_mutex> my_successors;
|
||||
}; //indexer_node_base
|
||||
|
||||
|
||||
template<int N, typename InputTuple> struct input_types;
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<1, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type > type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<2, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<3, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<4, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<5, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<6, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename tuple_element<5, InputTuple>::type sixth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type, sixth_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<7, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename tuple_element<5, InputTuple>::type sixth_type;
|
||||
typedef typename tuple_element<6, InputTuple>::type seventh_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type, sixth_type,
|
||||
seventh_type> type;
|
||||
};
|
||||
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<8, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename tuple_element<5, InputTuple>::type sixth_type;
|
||||
typedef typename tuple_element<6, InputTuple>::type seventh_type;
|
||||
typedef typename tuple_element<7, InputTuple>::type eighth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type, sixth_type,
|
||||
seventh_type, eighth_type> type;
|
||||
};
|
||||
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<9, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename tuple_element<5, InputTuple>::type sixth_type;
|
||||
typedef typename tuple_element<6, InputTuple>::type seventh_type;
|
||||
typedef typename tuple_element<7, InputTuple>::type eighth_type;
|
||||
typedef typename tuple_element<8, InputTuple>::type nineth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type, sixth_type,
|
||||
seventh_type, eighth_type, nineth_type> type;
|
||||
};
|
||||
|
||||
template<typename InputTuple>
|
||||
struct input_types<10, InputTuple> {
|
||||
typedef typename tuple_element<0, InputTuple>::type first_type;
|
||||
typedef typename tuple_element<1, InputTuple>::type second_type;
|
||||
typedef typename tuple_element<2, InputTuple>::type third_type;
|
||||
typedef typename tuple_element<3, InputTuple>::type fourth_type;
|
||||
typedef typename tuple_element<4, InputTuple>::type fifth_type;
|
||||
typedef typename tuple_element<5, InputTuple>::type sixth_type;
|
||||
typedef typename tuple_element<6, InputTuple>::type seventh_type;
|
||||
typedef typename tuple_element<7, InputTuple>::type eighth_type;
|
||||
typedef typename tuple_element<8, InputTuple>::type nineth_type;
|
||||
typedef typename tuple_element<9, InputTuple>::type tenth_type;
|
||||
typedef typename internal::tagged_msg<size_t, first_type, second_type, third_type,
|
||||
fourth_type, fifth_type, sixth_type,
|
||||
seventh_type, eighth_type, nineth_type,
|
||||
tenth_type> type;
|
||||
};
|
||||
|
||||
// type generators
|
||||
template<typename OutputTuple>
|
||||
struct indexer_types : public input_types<tuple_size<OutputTuple>::value, OutputTuple> {
|
||||
static const int N = tbb::flow::tuple_size<OutputTuple>::value;
|
||||
typedef typename input_types<N, OutputTuple>::type output_type;
|
||||
typedef typename wrap_tuple_elements<N,indexer_input_port,OutputTuple>::type input_ports_type;
|
||||
typedef internal::indexer_node_FE<input_ports_type,output_type,OutputTuple> indexer_FE_type;
|
||||
typedef internal::indexer_node_base<input_ports_type, output_type, OutputTuple> indexer_base_type;
|
||||
};
|
||||
|
||||
template<class OutputTuple>
|
||||
class unfolded_indexer_node : public indexer_types<OutputTuple>::indexer_base_type {
|
||||
public:
|
||||
typedef typename indexer_types<OutputTuple>::input_ports_type input_ports_type;
|
||||
typedef OutputTuple tuple_types;
|
||||
typedef typename indexer_types<OutputTuple>::output_type output_type;
|
||||
private:
|
||||
typedef typename indexer_types<OutputTuple>::indexer_base_type base_type;
|
||||
public:
|
||||
unfolded_indexer_node(graph& g) : base_type(g) {}
|
||||
unfolded_indexer_node(const unfolded_indexer_node &other) : base_type(other) {}
|
||||
};
|
||||
|
||||
} /* namespace internal */
|
||||
|
||||
#endif /* __TBB__flow_graph_indexer_impl_H */
|
||||
@@ -0,0 +1,279 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__flow_graph_item_buffer_impl_H
|
||||
#define __TBB__flow_graph_item_buffer_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#include "tbb/internal/_flow_graph_types_impl.h" // for aligned_pair
|
||||
|
||||
// in namespace tbb::flow::interface7 (included in _flow_graph_node_impl.h)
|
||||
|
||||
//! Expandable buffer of items. The possible operations are push, pop,
|
||||
//* tests for empty and so forth. No mutual exclusion is built in.
|
||||
//* objects are constructed into and explicitly-destroyed. get_my_item gives
|
||||
// a read-only reference to the item in the buffer. set_my_item may be called
|
||||
// with either an empty or occupied slot.
|
||||
|
||||
using internal::aligned_pair;
|
||||
using internal::alignment_of;
|
||||
|
||||
namespace internal {
|
||||
|
||||
template <typename T, typename A=cache_aligned_allocator<T> >
|
||||
class item_buffer {
|
||||
public:
|
||||
typedef T item_type;
|
||||
enum buffer_item_state { no_item=0, has_item=1, reserved_item=2 };
|
||||
protected:
|
||||
typedef size_t size_type;
|
||||
typedef typename aligned_pair<item_type, buffer_item_state>::type buffer_item_type;
|
||||
typedef typename A::template rebind<buffer_item_type>::other allocator_type;
|
||||
|
||||
buffer_item_type *my_array;
|
||||
size_type my_array_size;
|
||||
static const size_type initial_buffer_size = 4;
|
||||
size_type my_head;
|
||||
size_type my_tail;
|
||||
|
||||
bool buffer_empty() { return my_head == my_tail; }
|
||||
|
||||
buffer_item_type &item(size_type i) {
|
||||
__TBB_ASSERT(!(size_type(&(my_array[i&(my_array_size-1)].second))%alignment_of<buffer_item_state>::value),NULL);
|
||||
__TBB_ASSERT(!(size_type(&(my_array[i&(my_array_size-1)].first))%alignment_of<item_type>::value), NULL);
|
||||
return my_array[i & (my_array_size - 1) ];
|
||||
}
|
||||
|
||||
bool my_item_valid(size_type i) { return item(i).second != no_item; }
|
||||
bool my_item_reserved(size_type i) { return item(i).second == reserved_item; }
|
||||
|
||||
// object management in buffer
|
||||
const item_type &get_my_item(size_t i) {
|
||||
__TBB_ASSERT(my_item_valid(i),"attempt to get invalid item");
|
||||
item_type *itm = (tbb::internal::punned_cast<item_type *>(&(item(i).first)));
|
||||
return *(const item_type *)itm;
|
||||
}
|
||||
|
||||
// may be called with an empty slot or a slot that has already been constructed into.
|
||||
void set_my_item(size_t i, const item_type &o) {
|
||||
if(item(i).second != no_item) {
|
||||
destroy_item(i);
|
||||
}
|
||||
new(&(item(i).first)) item_type(o);
|
||||
item(i).second = has_item;
|
||||
}
|
||||
|
||||
// destructively-fetch an object from the buffer
|
||||
void fetch_item(size_t i, item_type &o) {
|
||||
__TBB_ASSERT(my_item_valid(i), "Trying to fetch an empty slot");
|
||||
o = get_my_item(i); // could have std::move assign semantics
|
||||
destroy_item(i);
|
||||
}
|
||||
|
||||
// move an existing item from one slot to another. The moved-to slot must be unoccupied,
|
||||
// the moved-from slot must exist and not be reserved. The after, from will be empty,
|
||||
// to will be occupied but not reserved
|
||||
void move_item(size_t to, size_t from) {
|
||||
__TBB_ASSERT(!my_item_valid(to), "Trying to move to a non-empty slot");
|
||||
__TBB_ASSERT(my_item_valid(from), "Trying to move from an empty slot");
|
||||
set_my_item(to, get_my_item(from)); // could have std::move semantics
|
||||
destroy_item(from);
|
||||
|
||||
}
|
||||
|
||||
// put an item in an empty slot. Return true if successful, else false
|
||||
bool place_item(size_t here, const item_type &me) {
|
||||
#if !TBB_DEPRECATED_SEQUENCER_DUPLICATES
|
||||
if(my_item_valid(here)) return false;
|
||||
#endif
|
||||
set_my_item(here, me);
|
||||
return true;
|
||||
}
|
||||
|
||||
// could be implemented with std::move semantics
|
||||
void swap_items(size_t i, size_t j) {
|
||||
__TBB_ASSERT(my_item_valid(i) && my_item_valid(j), "attempt to swap invalid item(s)");
|
||||
item_type temp = get_my_item(i);
|
||||
set_my_item(i, get_my_item(j));
|
||||
set_my_item(j, temp);
|
||||
}
|
||||
|
||||
void destroy_item(size_type i) {
|
||||
__TBB_ASSERT(my_item_valid(i), "destruction of invalid item");
|
||||
(tbb::internal::punned_cast<item_type *>(&(item(i).first)))->~item_type();
|
||||
item(i).second = no_item;
|
||||
}
|
||||
|
||||
// returns a copy of the front
|
||||
void copy_front(item_type &v) {
|
||||
__TBB_ASSERT(my_item_valid(my_head), "attempt to fetch head non-item");
|
||||
v = get_my_item(my_head);
|
||||
}
|
||||
// returns a copy of the back
|
||||
void copy_back(item_type &v) {
|
||||
__TBB_ASSERT(my_item_valid(my_tail-1), "attempt to fetch head non-item");
|
||||
v = get_my_item(my_tail-1);
|
||||
}
|
||||
|
||||
// following methods are for reservation of the front of a bufffer.
|
||||
void reserve_item(size_type i) { __TBB_ASSERT(my_item_valid(i) && !my_item_reserved(i), "item cannot be reserved"); item(i).second = reserved_item; }
|
||||
void release_item(size_type i) { __TBB_ASSERT(my_item_reserved(i), "item is not reserved"); item(i).second = has_item; }
|
||||
|
||||
void destroy_front() { destroy_item(my_head); ++my_head; }
|
||||
void destroy_back() { destroy_item(my_tail-1); --my_tail; }
|
||||
|
||||
// we have to be able to test against a new tail value without changing my_tail
|
||||
// grow_array doesn't work if we change my_tail when the old array is too small
|
||||
size_type size(size_t new_tail = 0) { return (new_tail ? new_tail : my_tail) - my_head; }
|
||||
size_type capacity() { return my_array_size; }
|
||||
// sequencer_node does not use this method, so we don't
|
||||
// need a version that passes in the new_tail value.
|
||||
bool buffer_full() { return size() >= capacity(); }
|
||||
|
||||
//! Grows the internal array.
|
||||
void grow_my_array( size_t minimum_size ) {
|
||||
// test that we haven't made the structure inconsistent.
|
||||
__TBB_ASSERT(capacity() >= my_tail - my_head, "total items exceed capacity");
|
||||
size_type new_size = my_array_size ? 2*my_array_size : initial_buffer_size;
|
||||
while( new_size<minimum_size )
|
||||
new_size*=2;
|
||||
|
||||
buffer_item_type* new_array = allocator_type().allocate(new_size);
|
||||
|
||||
// initialize validity to "no"
|
||||
for( size_type i=0; i<new_size; ++i ) { new_array[i].second = no_item; }
|
||||
|
||||
for( size_type i=my_head; i<my_tail; ++i) {
|
||||
if(my_item_valid(i)) { // sequencer_node may have empty slots
|
||||
// placement-new copy-construct; could be std::move
|
||||
char *new_space = (char *)&(new_array[i&(new_size-1)].first);
|
||||
(void)new(new_space) item_type(get_my_item(i));
|
||||
new_array[i&(new_size-1)].second = item(i).second;
|
||||
}
|
||||
}
|
||||
|
||||
clean_up_buffer(/*reset_pointers*/false);
|
||||
|
||||
my_array = new_array;
|
||||
my_array_size = new_size;
|
||||
}
|
||||
|
||||
bool push_back(item_type &v) {
|
||||
if(buffer_full()) {
|
||||
grow_my_array(size() + 1);
|
||||
}
|
||||
set_my_item(my_tail, v);
|
||||
++my_tail;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool pop_back(item_type &v) {
|
||||
if (!my_item_valid(my_tail-1)) {
|
||||
return false;
|
||||
}
|
||||
copy_back(v);
|
||||
destroy_back();
|
||||
return true;
|
||||
}
|
||||
|
||||
bool pop_front(item_type &v) {
|
||||
if(!my_item_valid(my_head)) {
|
||||
return false;
|
||||
}
|
||||
copy_front(v);
|
||||
destroy_front();
|
||||
return true;
|
||||
}
|
||||
|
||||
// This is used both for reset and for grow_my_array. In the case of grow_my_array
|
||||
// we want to retain the values of the head and tail.
|
||||
void clean_up_buffer(bool reset_pointers) {
|
||||
if (my_array) {
|
||||
for( size_type i=0; i<my_array_size; ++i ) {
|
||||
if(my_item_valid(i))
|
||||
destroy_item(i);
|
||||
}
|
||||
allocator_type().deallocate(my_array,my_array_size);
|
||||
}
|
||||
my_array = NULL;
|
||||
if(reset_pointers) {
|
||||
my_head = my_tail = my_array_size = 0;
|
||||
}
|
||||
}
|
||||
|
||||
public:
|
||||
//! Constructor
|
||||
item_buffer( ) : my_array(NULL), my_array_size(0),
|
||||
my_head(0), my_tail(0) {
|
||||
grow_my_array(initial_buffer_size);
|
||||
}
|
||||
|
||||
~item_buffer() {
|
||||
clean_up_buffer(/*reset_pointers*/true);
|
||||
}
|
||||
|
||||
void reset() { clean_up_buffer(/*reset_pointers*/true); grow_my_array(initial_buffer_size); }
|
||||
|
||||
};
|
||||
|
||||
//! item_buffer with reservable front-end. NOTE: if reserving, do not
|
||||
//* complete operation with pop_front(); use consume_front().
|
||||
//* No synchronization built-in.
|
||||
template<typename T, typename A=cache_aligned_allocator<T> >
|
||||
class reservable_item_buffer : public item_buffer<T, A> {
|
||||
protected:
|
||||
using item_buffer<T, A>::my_item_valid;
|
||||
using item_buffer<T, A>::my_head;
|
||||
|
||||
public:
|
||||
reservable_item_buffer() : item_buffer<T, A>(), my_reserved(false) {}
|
||||
void reset() {my_reserved = false; item_buffer<T,A>::reset(); }
|
||||
protected:
|
||||
|
||||
bool reserve_front(T &v) {
|
||||
if(my_reserved || !my_item_valid(my_head)) return false;
|
||||
my_reserved = true;
|
||||
// reserving the head
|
||||
this->copy_front(v);
|
||||
this->reserve_item(this->my_head);
|
||||
return true;
|
||||
}
|
||||
|
||||
void consume_front() {
|
||||
__TBB_ASSERT(my_reserved, "Attempt to consume a non-reserved item");
|
||||
this->destroy_front();
|
||||
my_reserved = false;
|
||||
}
|
||||
|
||||
void release_front() {
|
||||
__TBB_ASSERT(my_reserved, "Attempt to release a non-reserved item");
|
||||
this->release_item(this->my_head);
|
||||
my_reserved = false;
|
||||
}
|
||||
|
||||
bool my_reserved;
|
||||
};
|
||||
|
||||
} // namespace internal
|
||||
|
||||
#endif // __TBB__flow_graph_item_buffer_impl_H
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,742 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__flow_graph_node_impl_H
|
||||
#define __TBB__flow_graph_node_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#include "_flow_graph_item_buffer_impl.h"
|
||||
|
||||
//! @cond INTERNAL
|
||||
namespace internal {
|
||||
|
||||
using tbb::internal::aggregated_operation;
|
||||
using tbb::internal::aggregating_functor;
|
||||
using tbb::internal::aggregator;
|
||||
|
||||
template< typename T, typename A >
|
||||
class function_input_queue : public item_buffer<T,A> {
|
||||
public:
|
||||
bool pop( T& t ) {
|
||||
return this->pop_front( t );
|
||||
}
|
||||
|
||||
bool push( T& t ) {
|
||||
return this->push_back( t );
|
||||
}
|
||||
};
|
||||
|
||||
//! Input and scheduling for a function node that takes a type Input as input
|
||||
// The only up-ref is apply_body_impl, which should implement the function
|
||||
// call and any handling of the result.
|
||||
template< typename Input, typename A, typename ImplType >
|
||||
class function_input_base : public receiver<Input>, tbb::internal::no_assign {
|
||||
enum op_stat {WAIT=0, SUCCEEDED, FAILED};
|
||||
enum op_type {reg_pred, rem_pred, app_body, try_fwd, tryput_bypass, app_body_bypass
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
, add_blt_pred, del_blt_pred,
|
||||
blt_pred_cnt, blt_pred_cpy // create vector copies of preds and succs
|
||||
#endif
|
||||
};
|
||||
typedef function_input_base<Input, A, ImplType> my_class;
|
||||
|
||||
public:
|
||||
|
||||
//! The input type of this receiver
|
||||
typedef Input input_type;
|
||||
typedef sender<Input> predecessor_type;
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<predecessor_type *> predecessor_vector_type;
|
||||
#endif
|
||||
|
||||
//! Constructor for function_input_base
|
||||
function_input_base( graph &g, size_t max_concurrency, function_input_queue<input_type,A> *q = NULL )
|
||||
: my_graph(g), my_max_concurrency(max_concurrency), my_concurrency(0),
|
||||
my_queue(q), forwarder_busy(false) {
|
||||
my_predecessors.set_owner(this);
|
||||
my_aggregator.initialize_handler(my_handler(this));
|
||||
}
|
||||
|
||||
//! Copy constructor
|
||||
function_input_base( const function_input_base& src, function_input_queue<input_type,A> *q = NULL ) :
|
||||
receiver<Input>(), tbb::internal::no_assign(),
|
||||
my_graph(src.my_graph), my_max_concurrency(src.my_max_concurrency),
|
||||
my_concurrency(0), my_queue(q), forwarder_busy(false)
|
||||
{
|
||||
my_predecessors.set_owner(this);
|
||||
my_aggregator.initialize_handler(my_handler(this));
|
||||
}
|
||||
|
||||
//! Destructor
|
||||
virtual ~function_input_base() {
|
||||
if ( my_queue ) delete my_queue;
|
||||
}
|
||||
|
||||
//! Put to the node, returning a task if available
|
||||
virtual task * try_put_task( const input_type &t ) {
|
||||
if ( my_max_concurrency == 0 ) {
|
||||
return create_body_task( t );
|
||||
} else {
|
||||
my_operation op_data(t, tryput_bypass);
|
||||
my_aggregator.execute(&op_data);
|
||||
if(op_data.status == SUCCEEDED ) {
|
||||
return op_data.bypass_t;
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
//! Adds src to the list of cached predecessors.
|
||||
/* override */ bool register_predecessor( predecessor_type &src ) {
|
||||
my_operation op_data(reg_pred);
|
||||
op_data.r = &src;
|
||||
my_aggregator.execute(&op_data);
|
||||
return true;
|
||||
}
|
||||
|
||||
//! Removes src from the list of cached predecessors.
|
||||
/* override */ bool remove_predecessor( predecessor_type &src ) {
|
||||
my_operation op_data(rem_pred);
|
||||
op_data.r = &src;
|
||||
my_aggregator.execute(&op_data);
|
||||
return true;
|
||||
}
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
//! Adds to list of predecessors added by make_edge
|
||||
/*override*/ void internal_add_built_predecessor( predecessor_type &src) {
|
||||
my_operation op_data(add_blt_pred);
|
||||
op_data.r = &src;
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
|
||||
//! removes from to list of predecessors (used by remove_edge)
|
||||
/*override*/ void internal_delete_built_predecessor( predecessor_type &src) {
|
||||
my_operation op_data(del_blt_pred);
|
||||
op_data.r = &src;
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
|
||||
/*override*/ size_t predecessor_count() {
|
||||
my_operation op_data(blt_pred_cnt);
|
||||
my_aggregator.execute(&op_data);
|
||||
return op_data.cnt_val;
|
||||
}
|
||||
|
||||
/*override*/ void copy_predecessors(predecessor_vector_type &v) {
|
||||
my_operation op_data(blt_pred_cpy);
|
||||
op_data.predv = &v;
|
||||
my_aggregator.execute(&op_data);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
|
||||
protected:
|
||||
|
||||
void reset_function_input_base( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
my_concurrency = 0;
|
||||
if(my_queue) {
|
||||
my_queue->reset();
|
||||
}
|
||||
reset_receiver(__TBB_PFG_RESET_ARG(f));
|
||||
forwarder_busy = false;
|
||||
}
|
||||
|
||||
graph& my_graph;
|
||||
const size_t my_max_concurrency;
|
||||
size_t my_concurrency;
|
||||
function_input_queue<input_type, A> *my_queue;
|
||||
predecessor_cache<input_type, null_mutex > my_predecessors;
|
||||
|
||||
/*override*/void reset_receiver( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
my_predecessors.reset(__TBB_PFG_RESET_ARG(f));
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
__TBB_ASSERT(!(f & rf_extract) || my_predecessors.empty(), "function_input_base reset failed");
|
||||
#endif
|
||||
}
|
||||
|
||||
private:
|
||||
|
||||
friend class apply_body_task_bypass< my_class, input_type >;
|
||||
friend class forward_task_bypass< my_class >;
|
||||
|
||||
class my_operation : public aggregated_operation< my_operation > {
|
||||
public:
|
||||
char type;
|
||||
union {
|
||||
input_type *elem;
|
||||
predecessor_type *r;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
size_t cnt_val;
|
||||
predecessor_vector_type *predv;
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
};
|
||||
tbb::task *bypass_t;
|
||||
my_operation(const input_type& e, op_type t) :
|
||||
type(char(t)), elem(const_cast<input_type*>(&e)) {}
|
||||
my_operation(op_type t) : type(char(t)), r(NULL) {}
|
||||
};
|
||||
|
||||
bool forwarder_busy;
|
||||
typedef internal::aggregating_functor<my_class, my_operation> my_handler;
|
||||
friend class internal::aggregating_functor<my_class, my_operation>;
|
||||
aggregator< my_handler, my_operation > my_aggregator;
|
||||
|
||||
void handle_operations(my_operation *op_list) {
|
||||
my_operation *tmp;
|
||||
while (op_list) {
|
||||
tmp = op_list;
|
||||
op_list = op_list->next;
|
||||
switch (tmp->type) {
|
||||
case reg_pred:
|
||||
my_predecessors.add(*(tmp->r));
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
if (!forwarder_busy) {
|
||||
forwarder_busy = true;
|
||||
spawn_forward_task();
|
||||
}
|
||||
break;
|
||||
case rem_pred:
|
||||
my_predecessors.remove(*(tmp->r));
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
break;
|
||||
case app_body:
|
||||
__TBB_ASSERT(my_max_concurrency != 0, NULL);
|
||||
--my_concurrency;
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
if (my_concurrency<my_max_concurrency) {
|
||||
input_type i;
|
||||
bool item_was_retrieved = false;
|
||||
if ( my_queue )
|
||||
item_was_retrieved = my_queue->pop(i);
|
||||
else
|
||||
item_was_retrieved = my_predecessors.get_item(i);
|
||||
if (item_was_retrieved) {
|
||||
++my_concurrency;
|
||||
spawn_body_task(i);
|
||||
}
|
||||
}
|
||||
break;
|
||||
case app_body_bypass: {
|
||||
task * new_task = NULL;
|
||||
__TBB_ASSERT(my_max_concurrency != 0, NULL);
|
||||
--my_concurrency;
|
||||
if (my_concurrency<my_max_concurrency) {
|
||||
input_type i;
|
||||
bool item_was_retrieved = false;
|
||||
if ( my_queue )
|
||||
item_was_retrieved = my_queue->pop(i);
|
||||
else
|
||||
item_was_retrieved = my_predecessors.get_item(i);
|
||||
if (item_was_retrieved) {
|
||||
++my_concurrency;
|
||||
new_task = create_body_task(i);
|
||||
}
|
||||
}
|
||||
tmp->bypass_t = new_task;
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
}
|
||||
break;
|
||||
case tryput_bypass: internal_try_put_task(tmp); break;
|
||||
case try_fwd: internal_forward(tmp); break;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
case add_blt_pred: {
|
||||
my_predecessors.internal_add_built_predecessor(*(tmp->r));
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
}
|
||||
break;
|
||||
case del_blt_pred:
|
||||
my_predecessors.internal_delete_built_predecessor(*(tmp->r));
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
break;
|
||||
case blt_pred_cnt:
|
||||
tmp->cnt_val = my_predecessors.predecessor_count();
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
break;
|
||||
case blt_pred_cpy:
|
||||
my_predecessors.copy_predecessors( *(tmp->predv) );
|
||||
__TBB_store_with_release(tmp->status, SUCCEEDED);
|
||||
break;
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//! Put to the node, but return the task instead of enqueueing it
|
||||
void internal_try_put_task(my_operation *op) {
|
||||
__TBB_ASSERT(my_max_concurrency != 0, NULL);
|
||||
if (my_concurrency < my_max_concurrency) {
|
||||
++my_concurrency;
|
||||
task * new_task = create_body_task(*(op->elem));
|
||||
op->bypass_t = new_task;
|
||||
__TBB_store_with_release(op->status, SUCCEEDED);
|
||||
} else if ( my_queue && my_queue->push(*(op->elem)) ) {
|
||||
op->bypass_t = SUCCESSFULLY_ENQUEUED;
|
||||
__TBB_store_with_release(op->status, SUCCEEDED);
|
||||
} else {
|
||||
op->bypass_t = NULL;
|
||||
__TBB_store_with_release(op->status, FAILED);
|
||||
}
|
||||
}
|
||||
|
||||
//! Tries to spawn bodies if available and if concurrency allows
|
||||
void internal_forward(my_operation *op) {
|
||||
op->bypass_t = NULL;
|
||||
if (my_concurrency<my_max_concurrency || !my_max_concurrency) {
|
||||
input_type i;
|
||||
bool item_was_retrieved = false;
|
||||
if ( my_queue )
|
||||
item_was_retrieved = my_queue->pop(i);
|
||||
else
|
||||
item_was_retrieved = my_predecessors.get_item(i);
|
||||
if (item_was_retrieved) {
|
||||
++my_concurrency;
|
||||
op->bypass_t = create_body_task(i);
|
||||
__TBB_store_with_release(op->status, SUCCEEDED);
|
||||
return;
|
||||
}
|
||||
}
|
||||
__TBB_store_with_release(op->status, FAILED);
|
||||
forwarder_busy = false;
|
||||
}
|
||||
|
||||
//! Applies the body to the provided input
|
||||
// then decides if more work is available
|
||||
void apply_body( input_type &i ) {
|
||||
task *new_task = apply_body_bypass(i);
|
||||
if(!new_task) return;
|
||||
if(new_task == SUCCESSFULLY_ENQUEUED) return;
|
||||
FLOW_SPAWN(*new_task);
|
||||
return;
|
||||
}
|
||||
|
||||
//! Applies the body to the provided input
|
||||
// then decides if more work is available
|
||||
task * apply_body_bypass( input_type &i ) {
|
||||
task * new_task = static_cast<ImplType *>(this)->apply_body_impl_bypass(i);
|
||||
if ( my_max_concurrency != 0 ) {
|
||||
my_operation op_data(app_body_bypass); // tries to pop an item or get_item, enqueues another apply_body
|
||||
my_aggregator.execute(&op_data);
|
||||
tbb::task *ttask = op_data.bypass_t;
|
||||
new_task = combine_tasks(new_task, ttask);
|
||||
}
|
||||
return new_task;
|
||||
}
|
||||
|
||||
//! allocates a task to call apply_body( input )
|
||||
inline task * create_body_task( const input_type &input ) {
|
||||
|
||||
task* tp = my_graph.root_task();
|
||||
return (tp) ?
|
||||
new(task::allocate_additional_child_of(*tp))
|
||||
apply_body_task_bypass < my_class, input_type >(*this, input) :
|
||||
NULL;
|
||||
}
|
||||
|
||||
//! Spawns a task that calls apply_body( input )
|
||||
inline void spawn_body_task( const input_type &input ) {
|
||||
task* tp = create_body_task(input);
|
||||
// tp == NULL => g.reset(), which shouldn't occur in concurrent context
|
||||
if(tp) {
|
||||
FLOW_SPAWN(*tp);
|
||||
}
|
||||
}
|
||||
|
||||
//! This is executed by an enqueued task, the "forwarder"
|
||||
task *forward_task() {
|
||||
my_operation op_data(try_fwd);
|
||||
task *rval = NULL;
|
||||
do {
|
||||
op_data.status = WAIT;
|
||||
my_aggregator.execute(&op_data);
|
||||
if(op_data.status == SUCCEEDED) {
|
||||
tbb::task *ttask = op_data.bypass_t;
|
||||
rval = combine_tasks(rval, ttask);
|
||||
}
|
||||
} while (op_data.status == SUCCEEDED);
|
||||
return rval;
|
||||
}
|
||||
|
||||
inline task *create_forward_task() {
|
||||
task* tp = my_graph.root_task();
|
||||
return (tp) ?
|
||||
new(task::allocate_additional_child_of(*tp)) forward_task_bypass< my_class >(*this) :
|
||||
NULL;
|
||||
}
|
||||
|
||||
//! Spawns a task that calls forward()
|
||||
inline void spawn_forward_task() {
|
||||
task* tp = create_forward_task();
|
||||
if(tp) {
|
||||
FLOW_SPAWN(*tp);
|
||||
}
|
||||
}
|
||||
}; // function_input_base
|
||||
|
||||
//! Implements methods for a function node that takes a type Input as input and sends
|
||||
// a type Output to its successors.
|
||||
template< typename Input, typename Output, typename A>
|
||||
class function_input : public function_input_base<Input, A, function_input<Input,Output,A> > {
|
||||
public:
|
||||
typedef Input input_type;
|
||||
typedef Output output_type;
|
||||
typedef function_input<Input,Output,A> my_class;
|
||||
typedef function_input_base<Input, A, my_class> base_type;
|
||||
typedef function_input_queue<input_type, A> input_queue_type;
|
||||
|
||||
|
||||
// constructor
|
||||
template<typename Body>
|
||||
function_input( graph &g, size_t max_concurrency, Body& body, function_input_queue<input_type,A> *q = NULL ) :
|
||||
base_type(g, max_concurrency, q),
|
||||
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) {
|
||||
}
|
||||
|
||||
//! Copy constructor
|
||||
function_input( const function_input& src, input_queue_type *q = NULL ) :
|
||||
base_type(src, q),
|
||||
my_body( src.my_body->clone() ) {
|
||||
}
|
||||
|
||||
~function_input() {
|
||||
delete my_body;
|
||||
}
|
||||
|
||||
template< typename Body >
|
||||
Body copy_function_object() {
|
||||
internal::function_body<input_type, output_type> &body_ref = *this->my_body;
|
||||
return dynamic_cast< internal::function_body_leaf<input_type, output_type, Body> & >(body_ref).get_body();
|
||||
}
|
||||
|
||||
task * apply_body_impl_bypass( const input_type &i) {
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
|
||||
// There is an extra copied needed to capture the
|
||||
// body execution without the try_put
|
||||
tbb::internal::fgt_begin_body( my_body );
|
||||
output_type v = (*my_body)(i);
|
||||
tbb::internal::fgt_end_body( my_body );
|
||||
task * new_task = successors().try_put_task( v );
|
||||
#else
|
||||
task * new_task = successors().try_put_task( (*my_body)(i) );
|
||||
#endif
|
||||
return new_task;
|
||||
}
|
||||
|
||||
protected:
|
||||
|
||||
void reset_function_input(__TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
base_type::reset_function_input_base(__TBB_PFG_RESET_ARG(f));
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
if(f & rf_reset_bodies) my_body->reset_body();
|
||||
#endif
|
||||
}
|
||||
|
||||
function_body<input_type, output_type> *my_body;
|
||||
virtual broadcast_cache<output_type > &successors() = 0;
|
||||
|
||||
}; // function_input
|
||||
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
// helper templates to reset the successor edges of the output ports of an multifunction_node
|
||||
template<int N>
|
||||
struct reset_element {
|
||||
template<typename P>
|
||||
static void reset_this(P &p, reset_flags f) {
|
||||
(void)tbb::flow::get<N-1>(p).successors().reset(f);
|
||||
reset_element<N-1>::reset_this(p, f);
|
||||
}
|
||||
template<typename P>
|
||||
static bool this_empty(P &p) {
|
||||
if(tbb::flow::get<N-1>(p).successors().empty())
|
||||
return reset_element<N-1>::this_empty(p);
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
template<>
|
||||
struct reset_element<1> {
|
||||
template<typename P>
|
||||
static void reset_this(P &p, reset_flags f) {
|
||||
(void)tbb::flow::get<0>(p).successors().reset(f);
|
||||
}
|
||||
template<typename P>
|
||||
static bool this_empty(P &p) {
|
||||
return tbb::flow::get<0>(p).successors().empty();
|
||||
}
|
||||
};
|
||||
#endif
|
||||
|
||||
//! Implements methods for a function node that takes a type Input as input
|
||||
// and has a tuple of output ports specified.
|
||||
template< typename Input, typename OutputPortSet, typename A>
|
||||
class multifunction_input : public function_input_base<Input, A, multifunction_input<Input,OutputPortSet,A> > {
|
||||
public:
|
||||
static const int N = tbb::flow::tuple_size<OutputPortSet>::value;
|
||||
typedef Input input_type;
|
||||
typedef OutputPortSet output_ports_type;
|
||||
typedef multifunction_input<Input,OutputPortSet,A> my_class;
|
||||
typedef function_input_base<Input, A, my_class> base_type;
|
||||
typedef function_input_queue<input_type, A> input_queue_type;
|
||||
|
||||
|
||||
// constructor
|
||||
template<typename Body>
|
||||
multifunction_input(
|
||||
graph &g,
|
||||
size_t max_concurrency,
|
||||
Body& body,
|
||||
function_input_queue<input_type,A> *q = NULL ) :
|
||||
base_type(g, max_concurrency, q),
|
||||
my_body( new internal::multifunction_body_leaf<input_type, output_ports_type, Body>(body) ) {
|
||||
}
|
||||
|
||||
//! Copy constructor
|
||||
multifunction_input( const multifunction_input& src, input_queue_type *q = NULL ) :
|
||||
base_type(src, q),
|
||||
my_body( src.my_body->clone() ) {
|
||||
}
|
||||
|
||||
~multifunction_input() {
|
||||
delete my_body;
|
||||
}
|
||||
|
||||
template< typename Body >
|
||||
Body copy_function_object() {
|
||||
internal::multifunction_body<input_type, output_ports_type> &body_ref = *this->my_body;
|
||||
return dynamic_cast< internal::multifunction_body_leaf<input_type, output_ports_type, Body> & >(body_ref).get_body();
|
||||
}
|
||||
|
||||
// for multifunction nodes we do not have a single successor as such. So we just tell
|
||||
// the task we were successful.
|
||||
task * apply_body_impl_bypass( const input_type &i) {
|
||||
tbb::internal::fgt_begin_body( my_body );
|
||||
(*my_body)(i, my_output_ports);
|
||||
tbb::internal::fgt_end_body( my_body );
|
||||
task * new_task = SUCCESSFULLY_ENQUEUED;
|
||||
return new_task;
|
||||
}
|
||||
|
||||
output_ports_type &output_ports(){ return my_output_ports; }
|
||||
|
||||
protected:
|
||||
|
||||
/*override*/void reset(__TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
base_type::reset_function_input_base(__TBB_PFG_RESET_ARG(f));
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
reset_element<N>::reset_this(my_output_ports, f);
|
||||
if(f & rf_reset_bodies) my_body->reset_body();
|
||||
__TBB_ASSERT(!(f & rf_extract) || reset_element<N>::this_empty(my_output_ports), "multifunction_node reset failed");
|
||||
#endif
|
||||
}
|
||||
|
||||
multifunction_body<input_type, output_ports_type> *my_body;
|
||||
output_ports_type my_output_ports;
|
||||
|
||||
}; // multifunction_input
|
||||
|
||||
// template to refer to an output port of a multifunction_node
|
||||
template<size_t N, typename MOP>
|
||||
typename tbb::flow::tuple_element<N, typename MOP::output_ports_type>::type &output_port(MOP &op) {
|
||||
return tbb::flow::get<N>(op.output_ports());
|
||||
}
|
||||
|
||||
// helper structs for split_node
|
||||
template<int N>
|
||||
struct emit_element {
|
||||
template<typename T, typename P>
|
||||
static void emit_this(const T &t, P &p) {
|
||||
(void)tbb::flow::get<N-1>(p).try_put(tbb::flow::get<N-1>(t));
|
||||
emit_element<N-1>::emit_this(t,p);
|
||||
}
|
||||
};
|
||||
|
||||
template<>
|
||||
struct emit_element<1> {
|
||||
template<typename T, typename P>
|
||||
static void emit_this(const T &t, P &p) {
|
||||
(void)tbb::flow::get<0>(p).try_put(tbb::flow::get<0>(t));
|
||||
}
|
||||
};
|
||||
|
||||
//! Implements methods for an executable node that takes continue_msg as input
|
||||
template< typename Output >
|
||||
class continue_input : public continue_receiver {
|
||||
public:
|
||||
|
||||
//! The input type of this receiver
|
||||
typedef continue_msg input_type;
|
||||
|
||||
//! The output type of this receiver
|
||||
typedef Output output_type;
|
||||
|
||||
template< typename Body >
|
||||
continue_input( graph &g, Body& body )
|
||||
: my_graph_ptr(&g),
|
||||
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) { }
|
||||
|
||||
template< typename Body >
|
||||
continue_input( graph &g, int number_of_predecessors, Body& body )
|
||||
: continue_receiver( number_of_predecessors ), my_graph_ptr(&g),
|
||||
my_body( new internal::function_body_leaf< input_type, output_type, Body>(body) ) { }
|
||||
|
||||
continue_input( const continue_input& src ) : continue_receiver(src),
|
||||
my_graph_ptr(src.my_graph_ptr), my_body( src.my_body->clone() ) {}
|
||||
|
||||
~continue_input() {
|
||||
delete my_body;
|
||||
}
|
||||
|
||||
template< typename Body >
|
||||
Body copy_function_object() {
|
||||
internal::function_body<input_type, output_type> &body_ref = *my_body;
|
||||
return dynamic_cast< internal::function_body_leaf<input_type, output_type, Body> & >(body_ref).get_body();
|
||||
}
|
||||
|
||||
/*override*/void reset_receiver( __TBB_PFG_RESET_ARG(reset_flags f)) {
|
||||
continue_receiver::reset_receiver(__TBB_PFG_RESET_ARG(f));
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
if(f & rf_reset_bodies) my_body->reset_body();
|
||||
#endif
|
||||
}
|
||||
|
||||
protected:
|
||||
|
||||
graph* my_graph_ptr;
|
||||
function_body<input_type, output_type> *my_body;
|
||||
|
||||
virtual broadcast_cache<output_type > &successors() = 0;
|
||||
|
||||
friend class apply_body_task_bypass< continue_input< Output >, continue_msg >;
|
||||
|
||||
//! Applies the body to the provided input
|
||||
/* override */ task *apply_body_bypass( input_type ) {
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
|
||||
// There is an extra copied needed to capture the
|
||||
// body execution without the try_put
|
||||
tbb::internal::fgt_begin_body( my_body );
|
||||
output_type v = (*my_body)( continue_msg() );
|
||||
tbb::internal::fgt_end_body( my_body );
|
||||
return successors().try_put_task( v );
|
||||
#else
|
||||
return successors().try_put_task( (*my_body)( continue_msg() ) );
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Spawns a task that applies the body
|
||||
/* override */ task *execute( ) {
|
||||
task* tp = my_graph_ptr->root_task();
|
||||
return (tp) ?
|
||||
new ( task::allocate_additional_child_of( *tp ) )
|
||||
apply_body_task_bypass< continue_input< Output >, continue_msg >( *this, continue_msg() ) :
|
||||
NULL;
|
||||
}
|
||||
|
||||
}; // continue_input
|
||||
|
||||
//! Implements methods for both executable and function nodes that puts Output to its successors
|
||||
template< typename Output >
|
||||
class function_output : public sender<Output> {
|
||||
public:
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
template<int N> friend struct reset_element;
|
||||
#endif
|
||||
typedef Output output_type;
|
||||
typedef receiver<output_type> successor_type;
|
||||
typedef broadcast_cache<output_type> broadcast_cache_type;
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
typedef std::vector<successor_type *> successor_vector_type;
|
||||
#endif
|
||||
|
||||
function_output() { my_successors.set_owner(this); }
|
||||
function_output(const function_output & /*other*/) : sender<output_type>() {
|
||||
my_successors.set_owner(this);
|
||||
}
|
||||
|
||||
//! Adds a new successor to this node
|
||||
/* override */ bool register_successor( receiver<output_type> &r ) {
|
||||
successors().register_successor( r );
|
||||
return true;
|
||||
}
|
||||
|
||||
//! Removes a successor from this node
|
||||
/* override */ bool remove_successor( receiver<output_type> &r ) {
|
||||
successors().remove_successor( r );
|
||||
return true;
|
||||
}
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_FEATURES
|
||||
/*override*/ void internal_add_built_successor( receiver<output_type> &r) {
|
||||
successors().internal_add_built_successor( r );
|
||||
}
|
||||
|
||||
/*override*/ void internal_delete_built_successor( receiver<output_type> &r) {
|
||||
successors().internal_delete_built_successor( r );
|
||||
}
|
||||
|
||||
/*override*/ size_t successor_count() {
|
||||
return successors().successor_count();
|
||||
}
|
||||
|
||||
/*override*/ void copy_successors( successor_vector_type &v) {
|
||||
successors().copy_successors(v);
|
||||
}
|
||||
#endif /* TBB_PREVIEW_FLOW_GRAPH_FEATURES */
|
||||
|
||||
// for multifunction_node. The function_body that implements
|
||||
// the node will have an input and an output tuple of ports. To put
|
||||
// an item to a successor, the body should
|
||||
//
|
||||
// get<I>(output_ports).try_put(output_value);
|
||||
//
|
||||
// return value will be bool returned from successors.try_put.
|
||||
task *try_put_task(const output_type &i) { return my_successors.try_put_task(i); }
|
||||
|
||||
protected:
|
||||
broadcast_cache_type my_successors;
|
||||
broadcast_cache_type &successors() { return my_successors; }
|
||||
|
||||
}; // function_output
|
||||
|
||||
template< typename Output >
|
||||
class multifunction_output : public function_output<Output> {
|
||||
public:
|
||||
typedef Output output_type;
|
||||
typedef function_output<output_type> base_type;
|
||||
using base_type::my_successors;
|
||||
|
||||
multifunction_output() : base_type() {my_successors.set_owner(this);}
|
||||
multifunction_output( const multifunction_output &/*other*/) : base_type() { my_successors.set_owner(this); }
|
||||
|
||||
bool try_put(const output_type &i) {
|
||||
task *res = my_successors.try_put_task(i);
|
||||
if(!res) return false;
|
||||
if(res != SUCCESSFULLY_ENQUEUED) FLOW_SPAWN(*res);
|
||||
return true;
|
||||
}
|
||||
}; // multifunction_output
|
||||
|
||||
} // internal
|
||||
|
||||
#endif // __TBB__flow_graph_node_impl_H
|
||||
@@ -0,0 +1,251 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
// tagged buffer that can expand, and can support as many deletions as additions
|
||||
// list-based, with elements of list held in array (for destruction management),
|
||||
// multiplicative hashing (like ets). No synchronization built-in.
|
||||
//
|
||||
|
||||
#ifndef __TBB__flow_graph_tagged_buffer_impl_H
|
||||
#define __TBB__flow_graph_tagged_buffer_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
// included in namespace tbb::flow::interface7::internal
|
||||
|
||||
template<typename T, typename U, size_t NoTagMark>
|
||||
struct otherData {
|
||||
T t;
|
||||
U next;
|
||||
otherData() : t(NoTagMark), next(NULL) {}
|
||||
};
|
||||
|
||||
template<typename TagType, typename ValueType, size_t NoTagMark>
|
||||
struct buffer_element_type {
|
||||
// the second parameter below is void * because we can't forward-declare the type
|
||||
// itself, so we just reinterpret_cast below.
|
||||
typedef typename aligned_pair<ValueType, otherData<TagType, void *, NoTagMark> >::type type;
|
||||
};
|
||||
|
||||
template
|
||||
<
|
||||
typename TagType,
|
||||
typename ValueType,
|
||||
size_t NoTagMark = 0,
|
||||
typename Allocator=tbb::cache_aligned_allocator< typename buffer_element_type<TagType, ValueType, NoTagMark>::type >
|
||||
>
|
||||
class tagged_buffer {
|
||||
public:
|
||||
static const size_t INITIAL_SIZE = 8; // initial size of the hash pointer table
|
||||
static const TagType NO_TAG = TagType(NoTagMark);
|
||||
typedef ValueType value_type;
|
||||
typedef typename buffer_element_type<TagType, ValueType, NO_TAG>::type element_type;
|
||||
typedef value_type *pointer_type;
|
||||
typedef element_type *list_array_type; // array we manage manually
|
||||
typedef list_array_type *pointer_array_type;
|
||||
typedef typename Allocator::template rebind<list_array_type>::other pointer_array_allocator_type;
|
||||
typedef typename Allocator::template rebind<element_type>::other elements_array_allocator;
|
||||
private:
|
||||
size_t my_size;
|
||||
size_t nelements;
|
||||
pointer_array_type pointer_array; // pointer_array[my_size]
|
||||
list_array_type elements_array; // elements_array[my_size / 2]
|
||||
element_type* free_list;
|
||||
|
||||
size_t mask() { return my_size - 1; }
|
||||
|
||||
static size_t hash(TagType t) {
|
||||
return uintptr_t(t)*tbb::internal::select_size_t_constant<0x9E3779B9,0x9E3779B97F4A7C15ULL>::value;
|
||||
}
|
||||
|
||||
void set_up_free_list( element_type **p_free_list, list_array_type la, size_t sz) {
|
||||
for(size_t i=0; i < sz - 1; ++i ) { // construct free list
|
||||
la[i].second.next = &(la[i+1]);
|
||||
la[i].second.t = NO_TAG;
|
||||
}
|
||||
la[sz-1].second.next = NULL;
|
||||
*p_free_list = &(la[0]);
|
||||
}
|
||||
|
||||
// cleanup for exceptions
|
||||
struct DoCleanup {
|
||||
pointer_array_type *my_pa;
|
||||
list_array_type *my_elements;
|
||||
size_t my_size;
|
||||
|
||||
DoCleanup(pointer_array_type &pa, list_array_type &my_els, size_t sz) :
|
||||
my_pa(&pa), my_elements(&my_els), my_size(sz) { }
|
||||
~DoCleanup() {
|
||||
if(my_pa) {
|
||||
size_t dont_care = 0;
|
||||
internal_free_buffer(*my_pa, *my_elements, my_size, dont_care);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// exception-safety requires we do all the potentially-throwing operations first
|
||||
void grow_array() {
|
||||
size_t new_size = my_size*2;
|
||||
size_t new_nelements = nelements; // internal_free_buffer zeroes this
|
||||
list_array_type new_elements_array = NULL;
|
||||
pointer_array_type new_pointer_array = NULL;
|
||||
list_array_type new_free_list = NULL;
|
||||
{
|
||||
DoCleanup my_cleanup(new_pointer_array, new_elements_array, new_size);
|
||||
new_elements_array = elements_array_allocator().allocate(my_size);
|
||||
new_pointer_array = pointer_array_allocator_type().allocate(new_size);
|
||||
for(size_t i=0; i < new_size; ++i) new_pointer_array[i] = NULL;
|
||||
set_up_free_list(&new_free_list, new_elements_array, my_size );
|
||||
|
||||
for(size_t i=0; i < my_size; ++i) {
|
||||
for( element_type* op = pointer_array[i]; op; op = (element_type *)(op->second.next)) {
|
||||
value_type *ov = reinterpret_cast<value_type *>(&(op->first));
|
||||
// could have std::move semantics
|
||||
internal_tagged_insert(new_pointer_array, new_size, new_free_list, op->second.t, *ov);
|
||||
}
|
||||
}
|
||||
my_cleanup.my_pa = NULL;
|
||||
my_cleanup.my_elements = NULL;
|
||||
}
|
||||
|
||||
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
|
||||
free_list = new_free_list;
|
||||
pointer_array = new_pointer_array;
|
||||
elements_array = new_elements_array;
|
||||
my_size = new_size;
|
||||
nelements = new_nelements;
|
||||
}
|
||||
|
||||
// v should have perfect forwarding if std::move implemented.
|
||||
// we use this method to move elements in grow_array, so can't use class fields
|
||||
void internal_tagged_insert( element_type **p_pointer_array, size_t p_sz, list_array_type &p_free_list,
|
||||
const TagType t, const value_type &v) {
|
||||
size_t l_mask = p_sz-1;
|
||||
size_t h = hash(t) & l_mask;
|
||||
__TBB_ASSERT(p_free_list, "Error: free list not set up.");
|
||||
element_type* my_elem = p_free_list; p_free_list = (element_type *)(p_free_list->second.next);
|
||||
my_elem->second.t = t;
|
||||
(void) new(&(my_elem->first)) value_type(v);
|
||||
my_elem->second.next = p_pointer_array[h];
|
||||
p_pointer_array[h] = my_elem;
|
||||
}
|
||||
|
||||
void internal_initialize_buffer() {
|
||||
pointer_array = pointer_array_allocator_type().allocate(my_size);
|
||||
for(size_t i = 0; i < my_size; ++i) pointer_array[i] = NULL;
|
||||
elements_array = elements_array_allocator().allocate(my_size / 2);
|
||||
set_up_free_list(&free_list, elements_array, my_size / 2);
|
||||
}
|
||||
|
||||
// made static so an enclosed class can use to properly dispose of the internals
|
||||
static void internal_free_buffer( pointer_array_type &pa, list_array_type &el, size_t &sz, size_t &ne ) {
|
||||
if(pa) {
|
||||
for(size_t i = 0; i < sz; ++i ) {
|
||||
element_type *p_next;
|
||||
for( element_type *p = pa[i]; p; p = p_next) {
|
||||
p_next = (element_type *)p->second.next;
|
||||
value_type *vp = reinterpret_cast<value_type *>(&(p->first));
|
||||
vp->~value_type();
|
||||
}
|
||||
}
|
||||
pointer_array_allocator_type().deallocate(pa, sz);
|
||||
pa = NULL;
|
||||
}
|
||||
// Separate test (if allocation of pa throws, el may be allocated.
|
||||
// but no elements will be constructed.)
|
||||
if(el) {
|
||||
elements_array_allocator().deallocate(el, sz / 2);
|
||||
el = NULL;
|
||||
}
|
||||
sz = INITIAL_SIZE;
|
||||
ne = 0;
|
||||
}
|
||||
|
||||
public:
|
||||
tagged_buffer() : my_size(INITIAL_SIZE), nelements(0) {
|
||||
internal_initialize_buffer();
|
||||
}
|
||||
|
||||
~tagged_buffer() {
|
||||
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
|
||||
}
|
||||
|
||||
void reset() {
|
||||
internal_free_buffer(pointer_array, elements_array, my_size, nelements);
|
||||
internal_initialize_buffer();
|
||||
}
|
||||
|
||||
bool tagged_insert(const TagType t, const value_type &v) {
|
||||
pointer_type p;
|
||||
if(tagged_find_ref(t, p)) {
|
||||
p->~value_type();
|
||||
(void) new(p) value_type(v); // copy-construct into the space
|
||||
return false;
|
||||
}
|
||||
++nelements;
|
||||
if(nelements*2 > my_size) grow_array();
|
||||
internal_tagged_insert(pointer_array, my_size, free_list, t, v);
|
||||
return true;
|
||||
}
|
||||
|
||||
// returns reference to array element.v
|
||||
bool tagged_find_ref(const TagType t, pointer_type &v) {
|
||||
size_t i = hash(t) & mask();
|
||||
for(element_type* p = pointer_array[i]; p; p = (element_type *)(p->second.next)) {
|
||||
if(p->second.t == t) {
|
||||
v = reinterpret_cast<pointer_type>(&(p->first));
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool tagged_find( const TagType t, value_type &v) {
|
||||
value_type *p;
|
||||
if(tagged_find_ref(t, p)) {
|
||||
v = *p;
|
||||
return true;
|
||||
}
|
||||
else
|
||||
return false;
|
||||
}
|
||||
|
||||
void tagged_delete(const TagType t) {
|
||||
size_t h = hash(t) & mask();
|
||||
element_type* prev = NULL;
|
||||
for(element_type* p = pointer_array[h]; p; prev = p, p = (element_type *)(p->second.next)) {
|
||||
if(p->second.t == t) {
|
||||
value_type *vp = reinterpret_cast<value_type *>(&(p->first));
|
||||
vp->~value_type();
|
||||
p->second.t = NO_TAG;
|
||||
if(prev) prev->second.next = p->second.next;
|
||||
else pointer_array[h] = (element_type *)(p->second.next);
|
||||
p->second.next = free_list;
|
||||
free_list = p;
|
||||
--nelements;
|
||||
return;
|
||||
}
|
||||
}
|
||||
__TBB_ASSERT(false, "tag not found for delete");
|
||||
}
|
||||
};
|
||||
#endif // __TBB__flow_graph_tagged_buffer_impl_H
|
||||
@@ -0,0 +1,205 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _FGT_GRAPH_TRACE_IMPL_H
|
||||
#define _FGT_GRAPH_TRACE_IMPL_H
|
||||
|
||||
#include "../tbb_profiling.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
#if TBB_PREVIEW_FLOW_GRAPH_TRACE
|
||||
|
||||
static inline void fgt_internal_create_input_port( void *node, void *p, string_index name_index ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, p, FLOW_INPUT_PORT, node, FLOW_NODE, name_index );
|
||||
}
|
||||
|
||||
static inline void fgt_internal_create_output_port( void *node, void *p, string_index name_index ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, p, FLOW_OUTPUT_PORT, node, FLOW_NODE, name_index );
|
||||
}
|
||||
|
||||
template < typename TypesTuple, typename PortsTuple, int N >
|
||||
struct fgt_internal_input_helper {
|
||||
static void register_port( void *node, PortsTuple &ports ) {
|
||||
fgt_internal_create_input_port( node, (void*)static_cast< tbb::flow::interface7::receiver< typename tbb::flow::tuple_element<N-1,TypesTuple>::type > * >(&(tbb::flow::get<N-1>(ports))),
|
||||
static_cast<tbb::internal::string_index>(FLOW_INPUT_PORT_0 + N - 1) );
|
||||
fgt_internal_input_helper<TypesTuple, PortsTuple, N-1>::register_port( node, ports );
|
||||
}
|
||||
};
|
||||
|
||||
template < typename TypesTuple, typename PortsTuple >
|
||||
struct fgt_internal_input_helper<TypesTuple,PortsTuple,1> {
|
||||
static void register_port( void *node, PortsTuple &ports ) {
|
||||
fgt_internal_create_input_port( node, (void*)static_cast< tbb::flow::interface7::receiver< typename tbb::flow::tuple_element<0,TypesTuple>::type > * >(&(tbb::flow::get<0>(ports))),
|
||||
FLOW_INPUT_PORT_0 );
|
||||
}
|
||||
};
|
||||
|
||||
template < typename TypesTuple, typename PortsTuple, int N >
|
||||
struct fgt_internal_output_helper {
|
||||
static void register_port( void *node, PortsTuple &ports ) {
|
||||
fgt_internal_create_output_port( node, (void*)static_cast< tbb::flow::interface7::sender< typename tbb::flow::tuple_element<N-1,TypesTuple>::type > * >(&(tbb::flow::get<N-1>(ports))),
|
||||
static_cast<tbb::internal::string_index>(FLOW_OUTPUT_PORT_0 + N - 1) );
|
||||
fgt_internal_output_helper<TypesTuple, PortsTuple, N-1>::register_port( node, ports );
|
||||
}
|
||||
};
|
||||
|
||||
template < typename TypesTuple, typename PortsTuple >
|
||||
struct fgt_internal_output_helper<TypesTuple,PortsTuple,1> {
|
||||
static void register_port( void *node, PortsTuple &ports ) {
|
||||
fgt_internal_create_output_port( node, (void*)static_cast< tbb::flow::interface7::sender< typename tbb::flow::tuple_element<0,TypesTuple>::type > * >(&(tbb::flow::get<0>(ports))),
|
||||
FLOW_OUTPUT_PORT_0 );
|
||||
}
|
||||
};
|
||||
|
||||
template< typename NodeType >
|
||||
void fgt_multioutput_node_desc( const NodeType *node, const char *desc ) {
|
||||
void *addr = (void *)( static_cast< tbb::flow::interface7::receiver< typename NodeType::input_type > * >(const_cast< NodeType *>(node)) );
|
||||
itt_metadata_str_add( ITT_DOMAIN_FLOW, addr, FLOW_NODE, FLOW_OBJECT_NAME, desc );
|
||||
}
|
||||
|
||||
template< typename NodeType >
|
||||
static inline void fgt_node_desc( const NodeType *node, const char *desc ) {
|
||||
void *addr = (void *)( static_cast< tbb::flow::interface7::sender< typename NodeType::output_type > * >(const_cast< NodeType *>(node)) );
|
||||
itt_metadata_str_add( ITT_DOMAIN_FLOW, addr, FLOW_NODE, FLOW_OBJECT_NAME, desc );
|
||||
}
|
||||
|
||||
static inline void fgt_graph_desc( void *g, const char *desc ) {
|
||||
itt_metadata_str_add( ITT_DOMAIN_FLOW, g, FLOW_GRAPH, FLOW_OBJECT_NAME, desc );
|
||||
}
|
||||
|
||||
static inline void fgt_body( void *node, void *body ) {
|
||||
itt_relation_add( ITT_DOMAIN_FLOW, body, FLOW_BODY, __itt_relation_is_child_of, node, FLOW_NODE );
|
||||
}
|
||||
|
||||
template< typename OutputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multioutput_node( string_index t, void *g, void *input_port, PortsTuple &ports ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, input_port, FLOW_NODE, g, FLOW_GRAPH, t );
|
||||
fgt_internal_create_input_port( input_port, input_port, FLOW_INPUT_PORT_0 );
|
||||
fgt_internal_output_helper<OutputTuple, PortsTuple, N>::register_port( input_port, ports );
|
||||
}
|
||||
|
||||
template< typename OutputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multioutput_node_with_body( string_index t, void *g, void *input_port, PortsTuple &ports, void *body ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, input_port, FLOW_NODE, g, FLOW_GRAPH, t );
|
||||
fgt_internal_create_input_port( input_port, input_port, FLOW_INPUT_PORT_0 );
|
||||
fgt_internal_output_helper<OutputTuple, PortsTuple, N>::register_port( input_port, ports );
|
||||
fgt_body( input_port, body );
|
||||
}
|
||||
|
||||
|
||||
template< typename InputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multiinput_node( string_index t, void *g, PortsTuple &ports, void *output_port) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
|
||||
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
|
||||
fgt_internal_input_helper<InputTuple, PortsTuple, N>::register_port( output_port, ports );
|
||||
}
|
||||
|
||||
static inline void fgt_node( string_index t, void *g, void *output_port ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
|
||||
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
|
||||
}
|
||||
|
||||
static inline void fgt_node_with_body( string_index t, void *g, void *output_port, void *body ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, output_port, FLOW_NODE, g, FLOW_GRAPH, t );
|
||||
fgt_internal_create_output_port( output_port, output_port, FLOW_OUTPUT_PORT_0 );
|
||||
fgt_body( output_port, body );
|
||||
}
|
||||
|
||||
|
||||
static inline void fgt_node( string_index t, void *g, void *input_port, void *output_port ) {
|
||||
fgt_node( t, g, output_port );
|
||||
fgt_internal_create_input_port( output_port, input_port, FLOW_INPUT_PORT_0 );
|
||||
}
|
||||
|
||||
static inline void fgt_node_with_body( string_index t, void *g, void *input_port, void *output_port, void *body ) {
|
||||
fgt_node_with_body( t, g, output_port, body );
|
||||
fgt_internal_create_input_port( output_port, input_port, FLOW_INPUT_PORT_0 );
|
||||
}
|
||||
|
||||
|
||||
static inline void fgt_node( string_index t, void *g, void *input_port, void *decrement_port, void *output_port ) {
|
||||
fgt_node( t, g, input_port, output_port );
|
||||
fgt_internal_create_input_port( output_port, decrement_port, FLOW_INPUT_PORT_1 );
|
||||
}
|
||||
|
||||
static inline void fgt_make_edge( void *output_port, void *input_port ) {
|
||||
itt_relation_add( ITT_DOMAIN_FLOW, output_port, FLOW_OUTPUT_PORT, __itt_relation_is_predecessor_to, input_port, FLOW_INPUT_PORT);
|
||||
}
|
||||
|
||||
static inline void fgt_remove_edge( void *output_port, void *input_port ) {
|
||||
itt_relation_add( ITT_DOMAIN_FLOW, output_port, FLOW_OUTPUT_PORT, __itt_relation_is_sibling_of, input_port, FLOW_INPUT_PORT);
|
||||
}
|
||||
|
||||
static inline void fgt_graph( void *g ) {
|
||||
itt_make_task_group( ITT_DOMAIN_FLOW, g, FLOW_GRAPH, NULL, FLOW_NULL, FLOW_GRAPH );
|
||||
}
|
||||
|
||||
static inline void fgt_begin_body( void *body ) {
|
||||
itt_task_begin( ITT_DOMAIN_FLOW, body, FLOW_BODY, NULL, FLOW_NULL, FLOW_NULL );
|
||||
}
|
||||
|
||||
static inline void fgt_end_body( void * ) {
|
||||
itt_task_end( ITT_DOMAIN_FLOW );
|
||||
}
|
||||
|
||||
#else // TBB_PREVIEW_FLOW_GRAPH_TRACE
|
||||
|
||||
static inline void fgt_graph( void * /*g*/ ) { }
|
||||
|
||||
template< typename NodeType >
|
||||
static inline void fgt_multioutput_node_desc( const NodeType * /*node*/, const char * /*desc*/ ) { }
|
||||
|
||||
template< typename NodeType >
|
||||
static inline void fgt_node_desc( const NodeType * /*node*/, const char * /*desc*/ ) { }
|
||||
|
||||
static inline void fgt_graph_desc( void * /*g*/, const char * /*desc*/ ) { }
|
||||
|
||||
static inline void fgt_body( void * /*node*/, void * /*body*/ ) { }
|
||||
|
||||
template< typename OutputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multioutput_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, PortsTuple & /*ports*/ ) { }
|
||||
|
||||
template< typename OutputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multioutput_node_with_body( string_index /*t*/, void * /*g*/, void * /*input_port*/, PortsTuple & /*ports*/, void * /*body*/ ) { }
|
||||
|
||||
template< typename InputTuple, int N, typename PortsTuple >
|
||||
static inline void fgt_multiinput_node( string_index /*t*/, void * /*g*/, PortsTuple & /*ports*/, void * /*output_port*/ ) { }
|
||||
|
||||
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*output_port*/ ) { }
|
||||
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*output_port*/ ) { }
|
||||
static inline void fgt_node( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*decrement_port*/, void * /*output_port*/ ) { }
|
||||
|
||||
static inline void fgt_node_with_body( string_index /*t*/, void * /*g*/, void * /*output_port*/, void * /*body*/ ) { }
|
||||
static inline void fgt_node_with_body( string_index /*t*/, void * /*g*/, void * /*input_port*/, void * /*output_port*/, void * /*body*/ ) { }
|
||||
|
||||
static inline void fgt_make_edge( void * /*output_port*/, void * /*input_port*/ ) { }
|
||||
static inline void fgt_remove_edge( void * /*output_port*/, void * /*input_port*/ ) { }
|
||||
|
||||
static inline void fgt_begin_body( void * /*body*/ ) { }
|
||||
static inline void fgt_end_body( void * /*body*/) { }
|
||||
|
||||
#endif // TBB_PREVIEW_FLOW_GRAPH_TRACE
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,497 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__flow_graph_types_impl_H
|
||||
#define __TBB__flow_graph_types_impl_H
|
||||
|
||||
#ifndef __TBB_flow_graph_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
// included in namespace tbb::flow::interface7
|
||||
|
||||
namespace internal {
|
||||
// wrap each element of a tuple in a template, and make a tuple of the result.
|
||||
|
||||
template<int N, template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements;
|
||||
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<1, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<2, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<3, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<4, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<5, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
|
||||
#if __TBB_VARIADIC_MAX >= 6
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<6, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
#endif
|
||||
|
||||
#if __TBB_VARIADIC_MAX >= 7
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<7, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
#endif
|
||||
|
||||
#if __TBB_VARIADIC_MAX >= 8
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<8, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
#endif
|
||||
|
||||
#if __TBB_VARIADIC_MAX >= 9
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<9, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<8,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
#endif
|
||||
|
||||
#if __TBB_VARIADIC_MAX >= 10
|
||||
template<template<class> class PT, typename TypeTuple>
|
||||
struct wrap_tuple_elements<10, PT, TypeTuple> {
|
||||
typedef typename tbb::flow::tuple<
|
||||
PT<typename tbb::flow::tuple_element<0,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<1,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<2,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<3,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<4,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<5,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<6,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<7,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<8,TypeTuple>::type>,
|
||||
PT<typename tbb::flow::tuple_element<9,TypeTuple>::type> >
|
||||
type;
|
||||
};
|
||||
#endif
|
||||
|
||||
//! type mimicking std::pair but with trailing fill to ensure each element of an array
|
||||
//* will have the correct alignment
|
||||
template<typename T1, typename T2, size_t REM>
|
||||
struct type_plus_align {
|
||||
char first[sizeof(T1)];
|
||||
T2 second;
|
||||
char fill1[REM];
|
||||
};
|
||||
|
||||
template<typename T1, typename T2>
|
||||
struct type_plus_align<T1,T2,0> {
|
||||
char first[sizeof(T1)];
|
||||
T2 second;
|
||||
};
|
||||
|
||||
template<class U> struct alignment_of {
|
||||
typedef struct { char t; U padded; } test_alignment;
|
||||
static const size_t value = sizeof(test_alignment) - sizeof(U);
|
||||
};
|
||||
|
||||
// T1, T2 are actual types stored. The space defined for T1 in the type returned
|
||||
// is a char array of the correct size. Type T2 should be trivially-constructible,
|
||||
// T1 must be explicitly managed.
|
||||
template<typename T1, typename T2>
|
||||
struct aligned_pair {
|
||||
static const size_t t1_align = alignment_of<T1>::value;
|
||||
static const size_t t2_align = alignment_of<T2>::value;
|
||||
typedef type_plus_align<T1, T2, 0 > just_pair;
|
||||
static const size_t max_align = t1_align < t2_align ? t2_align : t1_align;
|
||||
static const size_t extra_bytes = sizeof(just_pair) % max_align;
|
||||
static const size_t remainder = extra_bytes ? max_align - extra_bytes : 0;
|
||||
public:
|
||||
typedef type_plus_align<T1,T2,remainder> type;
|
||||
}; // aligned_pair
|
||||
|
||||
// support for variant type
|
||||
// type we use when we're not storing a value
|
||||
struct default_constructed { };
|
||||
|
||||
// type which contains another type, tests for what type is contained, and references to it.
|
||||
// internal::Wrapper<T>
|
||||
// void CopyTo( void *newSpace) : builds a Wrapper<T> copy of itself in newSpace
|
||||
|
||||
// struct to allow us to copy and test the type of objects
|
||||
struct WrapperBase {
|
||||
virtual ~WrapperBase() {}
|
||||
virtual void CopyTo(void* /*newSpace*/) const { }
|
||||
};
|
||||
|
||||
// Wrapper<T> contains a T, with the ability to test what T is. The Wrapper<T> can be
|
||||
// constructed from a T, can be copy-constructed from another Wrapper<T>, and can be
|
||||
// examined via value(), but not modified.
|
||||
template<typename T>
|
||||
struct Wrapper: public WrapperBase {
|
||||
typedef T value_type;
|
||||
typedef T* pointer_type;
|
||||
private:
|
||||
T value_space;
|
||||
public:
|
||||
const value_type &value() const { return value_space; }
|
||||
|
||||
private:
|
||||
Wrapper();
|
||||
|
||||
// on exception will ensure the Wrapper will contain only a trivially-constructed object
|
||||
struct _unwind_space {
|
||||
pointer_type space;
|
||||
_unwind_space(pointer_type p) : space(p) {}
|
||||
~_unwind_space() {
|
||||
if(space) (void) new (space) Wrapper<default_constructed>(default_constructed());
|
||||
}
|
||||
};
|
||||
public:
|
||||
explicit Wrapper( const T& other ) : value_space(other) { }
|
||||
explicit Wrapper(const Wrapper& other) : value_space(other.value_space) { }
|
||||
|
||||
/*override*/void CopyTo(void* newSpace) const {
|
||||
_unwind_space guard((pointer_type)newSpace);
|
||||
(void) new(newSpace) Wrapper(value_space);
|
||||
guard.space = NULL;
|
||||
}
|
||||
/*override*/~Wrapper() { }
|
||||
};
|
||||
|
||||
// specialization for array objects
|
||||
template<typename T, size_t N>
|
||||
struct Wrapper<T[N]> : public WrapperBase {
|
||||
typedef T value_type;
|
||||
typedef T* pointer_type;
|
||||
// space must be untyped.
|
||||
typedef T ArrayType[N];
|
||||
private:
|
||||
// The space is not of type T[N] because when copy-constructing, it would be
|
||||
// default-initialized and then copied to in some fashion, resulting in two
|
||||
// constructions and one destruction per element. If the type is char[ ], we
|
||||
// placement new into each element, resulting in one construction per element.
|
||||
static const size_t space_size = sizeof(ArrayType) / sizeof(char);
|
||||
char value_space[space_size];
|
||||
|
||||
|
||||
// on exception will ensure the already-built objects will be destructed
|
||||
// (the value_space is a char array, so it is already trivially-destructible.)
|
||||
struct _unwind_class {
|
||||
pointer_type space;
|
||||
int already_built;
|
||||
_unwind_class(pointer_type p) : space(p), already_built(0) {}
|
||||
~_unwind_class() {
|
||||
if(space) {
|
||||
for(size_t i = already_built; i > 0 ; --i ) space[i-1].~value_type();
|
||||
(void) new(space) Wrapper<default_constructed>(default_constructed());
|
||||
}
|
||||
}
|
||||
};
|
||||
public:
|
||||
const ArrayType &value() const {
|
||||
char *vp = const_cast<char *>(value_space);
|
||||
return reinterpret_cast<ArrayType &>(*vp);
|
||||
}
|
||||
|
||||
private:
|
||||
Wrapper();
|
||||
public:
|
||||
// have to explicitly construct because other decays to a const value_type*
|
||||
explicit Wrapper(const ArrayType& other) {
|
||||
_unwind_class guard((pointer_type)value_space);
|
||||
pointer_type vp = reinterpret_cast<pointer_type>(&value_space);
|
||||
for(size_t i = 0; i < N; ++i ) {
|
||||
(void) new(vp++) value_type(other[i]);
|
||||
++(guard.already_built);
|
||||
}
|
||||
guard.space = NULL;
|
||||
}
|
||||
explicit Wrapper(const Wrapper& other) : WrapperBase() {
|
||||
// we have to do the heavy lifting to copy contents
|
||||
_unwind_class guard((pointer_type)value_space);
|
||||
pointer_type dp = reinterpret_cast<pointer_type>(value_space);
|
||||
pointer_type sp = reinterpret_cast<pointer_type>(const_cast<char *>(other.value_space));
|
||||
for(size_t i = 0; i < N; ++i, ++dp, ++sp) {
|
||||
(void) new(dp) value_type(*sp);
|
||||
++(guard.already_built);
|
||||
}
|
||||
guard.space = NULL;
|
||||
}
|
||||
|
||||
/*override*/void CopyTo(void* newSpace) const {
|
||||
(void) new(newSpace) Wrapper(*this); // exceptions handled in copy constructor
|
||||
}
|
||||
|
||||
/*override*/~Wrapper() {
|
||||
// have to destroy explicitly in reverse order
|
||||
pointer_type vp = reinterpret_cast<pointer_type>(&value_space);
|
||||
for(size_t i = N; i > 0 ; --i ) vp[i-1].~value_type();
|
||||
}
|
||||
};
|
||||
|
||||
// given a tuple, return the type of the element that has the maximum alignment requirement.
|
||||
// Given a tuple and that type, return the number of elements of the object with the max
|
||||
// alignment requirement that is at least as big as the largest object in the tuple.
|
||||
|
||||
template<bool, class T1, class T2> struct pick_one;
|
||||
template<class T1, class T2> struct pick_one<true , T1, T2> { typedef T1 type; };
|
||||
template<class T1, class T2> struct pick_one<false, T1, T2> { typedef T2 type; };
|
||||
|
||||
template< template<class> class Selector, typename T1, typename T2 >
|
||||
struct pick_max {
|
||||
typedef typename pick_one< (Selector<T1>::value > Selector<T2>::value), T1, T2 >::type type;
|
||||
};
|
||||
|
||||
template<typename T> struct size_of { static const int value = sizeof(T); };
|
||||
|
||||
template< size_t N, class Tuple, template<class> class Selector > struct pick_tuple_max {
|
||||
typedef typename pick_tuple_max<N-1, Tuple, Selector>::type LeftMaxType;
|
||||
typedef typename tbb::flow::tuple_element<N-1, Tuple>::type ThisType;
|
||||
typedef typename pick_max<Selector, LeftMaxType, ThisType>::type type;
|
||||
};
|
||||
|
||||
template< class Tuple, template<class> class Selector > struct pick_tuple_max<0, Tuple, Selector> {
|
||||
typedef typename tbb::flow::tuple_element<0, Tuple>::type type;
|
||||
};
|
||||
|
||||
// is the specified type included in a tuple?
|
||||
|
||||
template<class U, class V> struct is_same_type { static const bool value = false; };
|
||||
template<class W> struct is_same_type<W,W> { static const bool value = true; };
|
||||
|
||||
template<class Q, size_t N, class Tuple>
|
||||
struct is_element_of {
|
||||
typedef typename tbb::flow::tuple_element<N-1, Tuple>::type T_i;
|
||||
static const bool value = is_same_type<Q,T_i>::value || is_element_of<Q,N-1,Tuple>::value;
|
||||
};
|
||||
|
||||
template<class Q, class Tuple>
|
||||
struct is_element_of<Q,0,Tuple> {
|
||||
typedef typename tbb::flow::tuple_element<0, Tuple>::type T_i;
|
||||
static const bool value = is_same_type<Q,T_i>::value;
|
||||
};
|
||||
|
||||
// allow the construction of types that are listed tuple. If a disallowed type
|
||||
// construction is written, a method involving this type is created. The
|
||||
// type has no definition, so a syntax error is generated.
|
||||
template<typename T> struct ERROR_Type_Not_allowed_In_Tagged_Msg_Not_Member_Of_Tuple;
|
||||
|
||||
template<typename T, bool BUILD_IT> struct do_if;
|
||||
template<typename T>
|
||||
struct do_if<T, true> {
|
||||
static void construct(void *mySpace, const T& x) {
|
||||
(void) new(mySpace) Wrapper<T>(x);
|
||||
}
|
||||
};
|
||||
template<typename T>
|
||||
struct do_if<T, false> {
|
||||
static void construct(void * /*mySpace*/, const T& x) {
|
||||
// This method is instantiated when the type T does not match any of the
|
||||
// element types in the Tuple in variant<Tuple>.
|
||||
ERROR_Type_Not_allowed_In_Tagged_Msg_Not_Member_Of_Tuple<T>::bad_type(x);
|
||||
}
|
||||
};
|
||||
|
||||
// Tuple tells us the allowed types that variant can hold. It determines the alignment of the space in
|
||||
// Wrapper, and how big Wrapper is.
|
||||
//
|
||||
// the object can only be tested for type, and a read-only reference can be fetched by cast_to<T>().
|
||||
|
||||
using tbb::internal::punned_cast;
|
||||
struct tagged_null_type {};
|
||||
template<typename TagType, typename T0, typename T1=tagged_null_type, typename T2=tagged_null_type, typename T3=tagged_null_type,
|
||||
typename T4=tagged_null_type, typename T5=tagged_null_type, typename T6=tagged_null_type,
|
||||
typename T7=tagged_null_type, typename T8=tagged_null_type, typename T9=tagged_null_type>
|
||||
class tagged_msg {
|
||||
typedef tbb::flow::tuple<T0, T1, T2, T3, T4
|
||||
#if __TBB_VARIADIC_MAX >= 6
|
||||
, T5
|
||||
#endif
|
||||
#if __TBB_VARIADIC_MAX >= 7
|
||||
, T6
|
||||
#endif
|
||||
#if __TBB_VARIADIC_MAX >= 8
|
||||
, T7
|
||||
#endif
|
||||
#if __TBB_VARIADIC_MAX >= 9
|
||||
, T8
|
||||
#endif
|
||||
#if __TBB_VARIADIC_MAX >= 10
|
||||
, T9
|
||||
#endif
|
||||
> Tuple;
|
||||
|
||||
private:
|
||||
class variant {
|
||||
static const size_t N = tbb::flow::tuple_size<Tuple>::value;
|
||||
typedef typename pick_tuple_max<N, Tuple, alignment_of>::type AlignType;
|
||||
typedef typename pick_tuple_max<N, Tuple, size_of>::type MaxSizeType;
|
||||
static const size_t MaxNBytes = (sizeof(Wrapper<MaxSizeType>)+sizeof(AlignType)-1);
|
||||
static const size_t MaxNElements = MaxNBytes/sizeof(AlignType);
|
||||
typedef typename tbb::aligned_space<AlignType, MaxNElements> SpaceType;
|
||||
SpaceType my_space;
|
||||
static const size_t MaxSize = sizeof(SpaceType);
|
||||
|
||||
public:
|
||||
variant() { (void) new(&my_space) Wrapper<default_constructed>(default_constructed()); }
|
||||
|
||||
template<typename T>
|
||||
variant( const T& x ) {
|
||||
do_if<T, is_element_of<T, N, Tuple>::value>::construct(&my_space,x);
|
||||
}
|
||||
|
||||
variant(const variant& other) {
|
||||
const WrapperBase * h = punned_cast<const WrapperBase *>(&(other.my_space));
|
||||
h->CopyTo(&my_space);
|
||||
}
|
||||
|
||||
// assignment must destroy and re-create the Wrapper type, as there is no way
|
||||
// to create a Wrapper-to-Wrapper assign even if we find they agree in type.
|
||||
void operator=( const variant& rhs ) {
|
||||
if(&rhs != this) {
|
||||
WrapperBase *h = punned_cast<WrapperBase *>(&my_space);
|
||||
h->~WrapperBase();
|
||||
const WrapperBase *ch = punned_cast<const WrapperBase *>(&(rhs.my_space));
|
||||
ch->CopyTo(&my_space);
|
||||
}
|
||||
}
|
||||
|
||||
template<typename U>
|
||||
const U& variant_cast_to() const {
|
||||
const Wrapper<U> *h = dynamic_cast<const Wrapper<U>*>(punned_cast<const WrapperBase *>(&my_space));
|
||||
if(!h) {
|
||||
tbb::internal::throw_exception(tbb::internal::eid_bad_tagged_msg_cast);
|
||||
}
|
||||
return h->value();
|
||||
}
|
||||
template<typename U>
|
||||
bool variant_is_a() const { return dynamic_cast<const Wrapper<U>*>(punned_cast<const WrapperBase *>(&my_space)) != NULL; }
|
||||
|
||||
bool variant_is_default_constructed() const {return variant_is_a<default_constructed>();}
|
||||
|
||||
~variant() {
|
||||
WrapperBase *h = punned_cast<WrapperBase *>(&my_space);
|
||||
h->~WrapperBase();
|
||||
}
|
||||
}; //class variant
|
||||
|
||||
TagType my_tag;
|
||||
variant my_msg;
|
||||
|
||||
public:
|
||||
tagged_msg(): my_tag(TagType(~0)), my_msg(){}
|
||||
|
||||
template<typename T, typename R>
|
||||
tagged_msg(T const &index, R const &value) : my_tag(index), my_msg(value) {}
|
||||
|
||||
#if __TBB_CONST_REF_TO_ARRAY_TEMPLATE_PARAM_BROKEN
|
||||
template<typename T, typename R, size_t N>
|
||||
tagged_msg(T const &index, R (&value)[N]) : my_tag(index), my_msg(value) {}
|
||||
#endif
|
||||
|
||||
void set_tag(TagType const &index) {my_tag = index;}
|
||||
TagType tag() const {return my_tag;}
|
||||
|
||||
template<typename V>
|
||||
const V& cast_to() const {return my_msg.template variant_cast_to<V>();}
|
||||
|
||||
template<typename V>
|
||||
bool is_a() const {return my_msg.template variant_is_a<V>();}
|
||||
|
||||
bool is_default_constructed() const {return my_msg.variant_is_default_constructed();}
|
||||
}; //class tagged_msg
|
||||
|
||||
// template to simplify cast and test for tagged_msg in template contexts
|
||||
template<typename T, typename V>
|
||||
const T& cast_to(V const &v) { return v.template cast_to<T>(); }
|
||||
|
||||
template<typename T, typename V>
|
||||
bool is_a(V const &v) { return v.template is_a<T>(); }
|
||||
|
||||
} // namespace internal
|
||||
|
||||
#endif /* __TBB__flow_graph_types_impl_H */
|
||||
@@ -0,0 +1,102 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_mutex_padding_H
|
||||
#define __TBB_mutex_padding_H
|
||||
|
||||
// wrapper for padding mutexes to be alone on a cache line, without requiring they be allocated
|
||||
// from a pool. Because we allow them to be defined anywhere they must be two cache lines in size.
|
||||
|
||||
|
||||
namespace tbb {
|
||||
namespace interface7 {
|
||||
namespace internal {
|
||||
|
||||
static const size_t cache_line_size = 64;
|
||||
|
||||
// Pad a mutex to occupy a number of full cache lines sufficient to avoid false sharing
|
||||
// with other data; space overhead is up to 2*cache_line_size-1.
|
||||
template<typename Mutex, bool is_rw> class padded_mutex;
|
||||
|
||||
template<typename Mutex>
|
||||
class padded_mutex<Mutex,false> : tbb::internal::mutex_copy_deprecated_and_disabled {
|
||||
typedef long pad_type;
|
||||
pad_type my_pad[((sizeof(Mutex)+cache_line_size-1)/cache_line_size+1)*cache_line_size/sizeof(pad_type)];
|
||||
|
||||
Mutex *impl() { return (Mutex *)((uintptr_t(this)|(cache_line_size-1))+1);}
|
||||
|
||||
public:
|
||||
static const bool is_rw_mutex = Mutex::is_rw_mutex;
|
||||
static const bool is_recursive_mutex = Mutex::is_recursive_mutex;
|
||||
static const bool is_fair_mutex = Mutex::is_fair_mutex;
|
||||
|
||||
padded_mutex() { new(impl()) Mutex(); }
|
||||
~padded_mutex() { impl()->~Mutex(); }
|
||||
|
||||
//! Represents acquisition of a mutex.
|
||||
class scoped_lock : tbb::internal::no_copy {
|
||||
typename Mutex::scoped_lock my_scoped_lock;
|
||||
public:
|
||||
scoped_lock() : my_scoped_lock() {}
|
||||
scoped_lock( padded_mutex& m ) : my_scoped_lock(*m.impl()) { }
|
||||
~scoped_lock() { }
|
||||
|
||||
void acquire( padded_mutex& m ) { my_scoped_lock.acquire(*m.impl()); }
|
||||
bool try_acquire( padded_mutex& m ) { return my_scoped_lock.try_acquire(*m.impl()); }
|
||||
void release() { my_scoped_lock.release(); }
|
||||
};
|
||||
};
|
||||
|
||||
template<typename Mutex>
|
||||
class padded_mutex<Mutex,true> : tbb::internal::mutex_copy_deprecated_and_disabled {
|
||||
typedef long pad_type;
|
||||
pad_type my_pad[((sizeof(Mutex)+cache_line_size-1)/cache_line_size+1)*cache_line_size/sizeof(pad_type)];
|
||||
|
||||
Mutex *impl() { return (Mutex *)((uintptr_t(this)|(cache_line_size-1))+1);}
|
||||
|
||||
public:
|
||||
static const bool is_rw_mutex = Mutex::is_rw_mutex;
|
||||
static const bool is_recursive_mutex = Mutex::is_recursive_mutex;
|
||||
static const bool is_fair_mutex = Mutex::is_fair_mutex;
|
||||
|
||||
padded_mutex() { new(impl()) Mutex(); }
|
||||
~padded_mutex() { impl()->~Mutex(); }
|
||||
|
||||
//! Represents acquisition of a mutex.
|
||||
class scoped_lock : tbb::internal::no_copy {
|
||||
typename Mutex::scoped_lock my_scoped_lock;
|
||||
public:
|
||||
scoped_lock() : my_scoped_lock() {}
|
||||
scoped_lock( padded_mutex& m, bool write = true ) : my_scoped_lock(*m.impl(),write) { }
|
||||
~scoped_lock() { }
|
||||
|
||||
void acquire( padded_mutex& m, bool write = true ) { my_scoped_lock.acquire(*m.impl(),write); }
|
||||
bool try_acquire( padded_mutex& m, bool write = true ) { return my_scoped_lock.try_acquire(*m.impl(),write); }
|
||||
bool upgrade_to_writer() { return my_scoped_lock.upgrade_to_writer(); }
|
||||
bool downgrade_to_reader() { return my_scoped_lock.downgrade_to_reader(); }
|
||||
void release() { my_scoped_lock.release(); }
|
||||
};
|
||||
};
|
||||
|
||||
} // namespace internal
|
||||
} // namespace interface7
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_mutex_padding_H */
|
||||
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_range_iterator_H
|
||||
#define __TBB_range_iterator_H
|
||||
|
||||
#include "../tbb_stddef.h"
|
||||
|
||||
#if __TBB_CPP11_STD_BEGIN_END_PRESENT && __TBB_CPP11_AUTO_PRESENT && __TBB_CPP11_DECLTYPE_PRESENT
|
||||
#include <iterator>
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
// iterators to first and last elements of container
|
||||
namespace internal {
|
||||
|
||||
#if __TBB_CPP11_STD_BEGIN_END_PRESENT && __TBB_CPP11_AUTO_PRESENT && __TBB_CPP11_DECLTYPE_PRESENT
|
||||
using std::begin;
|
||||
using std::end;
|
||||
template<typename Container>
|
||||
auto first(Container& c)-> decltype(begin(c)) {return begin(c);}
|
||||
|
||||
template<typename Container>
|
||||
auto first(const Container& c)-> decltype(begin(c)) {return begin(c);}
|
||||
|
||||
template<typename Container>
|
||||
auto last(Container& c)-> decltype(begin(c)) {return end(c);}
|
||||
|
||||
template<typename Container>
|
||||
auto last(const Container& c)-> decltype(begin(c)) {return end(c);}
|
||||
#else
|
||||
template<typename Container>
|
||||
typename Container::iterator first(Container& c) {return c.begin();}
|
||||
|
||||
template<typename Container>
|
||||
typename Container::const_iterator first(const Container& c) {return c.begin();}
|
||||
|
||||
template<typename Container>
|
||||
typename Container::iterator last(Container& c) {return c.end();}
|
||||
|
||||
template<typename Container>
|
||||
typename Container::const_iterator last(const Container& c) {return c.end();}
|
||||
#endif
|
||||
|
||||
template<typename T, size_t size>
|
||||
T* first(T (&arr) [size]) {return arr;}
|
||||
|
||||
template<typename T, size_t size>
|
||||
T* last(T (&arr) [size]) {return arr + size;}
|
||||
} //namespace internal
|
||||
} //namespace tbb
|
||||
|
||||
#endif // __TBB_range_iterator_H
|
||||
@@ -0,0 +1,65 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
TBB_STRING_RESOURCE(FLOW_BROADCAST_NODE, "broadcast_node")
|
||||
TBB_STRING_RESOURCE(FLOW_BUFFER_NODE, "buffer_node")
|
||||
TBB_STRING_RESOURCE(FLOW_CONTINUE_NODE, "continue_node")
|
||||
TBB_STRING_RESOURCE(FLOW_FUNCTION_NODE, "function_node")
|
||||
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_QUEUEING, "join_node (queueing)")
|
||||
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_RESERVING, "join_node (reserving)")
|
||||
TBB_STRING_RESOURCE(FLOW_JOIN_NODE_TAG_MATCHING, "join_node (tag_matching)")
|
||||
TBB_STRING_RESOURCE(FLOW_LIMITER_NODE, "limiter_node")
|
||||
TBB_STRING_RESOURCE(FLOW_MULTIFUNCTION_NODE, "multifunction_node")
|
||||
TBB_STRING_RESOURCE(FLOW_OR_NODE, "or_node") //no longer in use, kept for backward compatibilty
|
||||
TBB_STRING_RESOURCE(FLOW_OVERWRITE_NODE, "overwrite_node")
|
||||
TBB_STRING_RESOURCE(FLOW_PRIORITY_QUEUE_NODE, "priority_queue_node")
|
||||
TBB_STRING_RESOURCE(FLOW_QUEUE_NODE, "queue_node")
|
||||
TBB_STRING_RESOURCE(FLOW_SEQUENCER_NODE, "sequencer_node")
|
||||
TBB_STRING_RESOURCE(FLOW_SOURCE_NODE, "source_node")
|
||||
TBB_STRING_RESOURCE(FLOW_SPLIT_NODE, "split_node")
|
||||
TBB_STRING_RESOURCE(FLOW_WRITE_ONCE_NODE, "write_once_node")
|
||||
TBB_STRING_RESOURCE(FLOW_BODY, "body")
|
||||
TBB_STRING_RESOURCE(FLOW_GRAPH, "graph")
|
||||
TBB_STRING_RESOURCE(FLOW_NODE, "node")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT, "input_port")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_0, "input_port_0")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_1, "input_port_1")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_2, "input_port_2")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_3, "input_port_3")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_4, "input_port_4")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_5, "input_port_5")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_6, "input_port_6")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_7, "input_port_7")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_8, "input_port_8")
|
||||
TBB_STRING_RESOURCE(FLOW_INPUT_PORT_9, "input_port_9")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT, "output_port")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_0, "output_port_0")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_1, "output_port_1")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_2, "output_port_2")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_3, "output_port_3")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_4, "output_port_4")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_5, "output_port_5")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_6, "output_port_6")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_7, "output_port_7")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_8, "output_port_8")
|
||||
TBB_STRING_RESOURCE(FLOW_OUTPUT_PORT_9, "output_port_9")
|
||||
TBB_STRING_RESOURCE(FLOW_OBJECT_NAME, "object_name")
|
||||
TBB_STRING_RESOURCE(FLOW_NULL, "null")
|
||||
TBB_STRING_RESOURCE(FLOW_INDEXER_NODE, "indexer_node")
|
||||
@@ -0,0 +1,73 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_tbb_windef_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif /* __TBB_tbb_windef_H */
|
||||
|
||||
// Check that the target Windows version has all API calls requried for TBB.
|
||||
// Do not increase the version in condition beyond 0x0500 without prior discussion!
|
||||
#if defined(_WIN32_WINNT) && _WIN32_WINNT<0x0501
|
||||
#error TBB is unable to run on old Windows versions; _WIN32_WINNT must be 0x0501 or greater.
|
||||
#endif
|
||||
|
||||
#if !defined(_MT)
|
||||
#error TBB requires linkage with multithreaded C/C++ runtime library. \
|
||||
Choose multithreaded DLL runtime in project settings, or use /MD[d] compiler switch.
|
||||
#endif
|
||||
|
||||
// Workaround for the problem with MVSC headers failing to define namespace std
|
||||
namespace std {
|
||||
using ::size_t; using ::ptrdiff_t;
|
||||
}
|
||||
|
||||
#define __TBB_STRING_AUX(x) #x
|
||||
#define __TBB_STRING(x) __TBB_STRING_AUX(x)
|
||||
|
||||
// Default setting of TBB_USE_DEBUG
|
||||
#ifdef TBB_USE_DEBUG
|
||||
# if TBB_USE_DEBUG
|
||||
# if !defined(_DEBUG)
|
||||
# pragma message(__FILE__ "(" __TBB_STRING(__LINE__) ") : Warning: Recommend using /MDd if compiling with TBB_USE_DEBUG!=0")
|
||||
# endif
|
||||
# else
|
||||
# if defined(_DEBUG)
|
||||
# pragma message(__FILE__ "(" __TBB_STRING(__LINE__) ") : Warning: Recommend using /MD if compiling with TBB_USE_DEBUG==0")
|
||||
# endif
|
||||
# endif
|
||||
#endif
|
||||
|
||||
#if (__TBB_BUILD || __TBBMALLOC_BUILD) && !defined(__TBB_NO_IMPLICIT_LINKAGE)
|
||||
#define __TBB_NO_IMPLICIT_LINKAGE 1
|
||||
#endif
|
||||
|
||||
#if _MSC_VER
|
||||
#if !__TBB_NO_IMPLICIT_LINKAGE
|
||||
#ifdef __TBB_LIB_NAME
|
||||
#pragma comment(lib, __TBB_STRING(__TBB_LIB_NAME))
|
||||
#else
|
||||
#ifdef _DEBUG
|
||||
#pragma comment(lib, "tbb_debug.lib")
|
||||
#else
|
||||
#pragma comment(lib, "tbb.lib")
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
#endif
|
||||
@@ -0,0 +1,148 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__x86_eliding_mutex_impl_H
|
||||
#define __TBB__x86_eliding_mutex_impl_H
|
||||
|
||||
#ifndef __TBB_spin_mutex_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#if ( __TBB_x86_32 || __TBB_x86_64 )
|
||||
|
||||
namespace tbb {
|
||||
namespace interface7 {
|
||||
namespace internal {
|
||||
|
||||
template<typename Mutex, bool is_rw>
|
||||
class padded_mutex;
|
||||
|
||||
//! An eliding lock that occupies a single byte.
|
||||
/** A x86_eliding_mutex is an HLE-enabled spin mutex. It is recommended to
|
||||
put the mutex on a cache line that is not shared by the data it protects.
|
||||
It should be used for locking short critical sections where the lock is
|
||||
contended but the data it protects are not. If zero-initialized, the
|
||||
mutex is considered unheld.
|
||||
@ingroup synchronization */
|
||||
class x86_eliding_mutex : tbb::internal::mutex_copy_deprecated_and_disabled {
|
||||
//! 0 if lock is released, 1 if lock is acquired.
|
||||
__TBB_atomic_flag flag;
|
||||
|
||||
friend class padded_mutex<x86_eliding_mutex, false>;
|
||||
|
||||
public:
|
||||
//! Construct unacquired lock.
|
||||
/** Equivalent to zero-initialization of *this. */
|
||||
x86_eliding_mutex() : flag(0) {}
|
||||
|
||||
// bug in gcc 3.x.x causes syntax error in spite of the friend declaration above.
|
||||
// Make the scoped_lock public in that case.
|
||||
#if __TBB_USE_X86_ELIDING_MUTEX || __TBB_GCC_VERSION < 40000
|
||||
#else
|
||||
// by default we will not provide the scoped_lock interface. The user
|
||||
// should use the padded version of the mutex. scoped_lock is used in
|
||||
// padded_mutex template.
|
||||
private:
|
||||
#endif
|
||||
// scoped_lock in padded_mutex<> is the interface to use.
|
||||
//! Represents acquisition of a mutex.
|
||||
class scoped_lock : tbb::internal::no_copy {
|
||||
private:
|
||||
//! Points to currently held mutex, or NULL if no lock is held.
|
||||
x86_eliding_mutex* my_mutex;
|
||||
|
||||
public:
|
||||
//! Construct without acquiring a mutex.
|
||||
scoped_lock() : my_mutex(NULL) {}
|
||||
|
||||
//! Construct and acquire lock on a mutex.
|
||||
scoped_lock( x86_eliding_mutex& m ) : my_mutex(NULL) { acquire(m); }
|
||||
|
||||
//! Acquire lock.
|
||||
void acquire( x86_eliding_mutex& m ) {
|
||||
__TBB_ASSERT( !my_mutex, "already holding a lock" );
|
||||
|
||||
my_mutex=&m;
|
||||
my_mutex->lock();
|
||||
}
|
||||
|
||||
//! Try acquiring lock (non-blocking)
|
||||
/** Return true if lock acquired; false otherwise. */
|
||||
bool try_acquire( x86_eliding_mutex& m ) {
|
||||
__TBB_ASSERT( !my_mutex, "already holding a lock" );
|
||||
|
||||
bool result = m.try_lock();
|
||||
if( result ) {
|
||||
my_mutex = &m;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
//! Release lock
|
||||
void release() {
|
||||
__TBB_ASSERT( my_mutex, "release on scoped_lock that is not holding a lock" );
|
||||
|
||||
my_mutex->unlock();
|
||||
my_mutex = NULL;
|
||||
}
|
||||
|
||||
//! Destroy lock. If holding a lock, releases the lock first.
|
||||
~scoped_lock() {
|
||||
if( my_mutex ) {
|
||||
release();
|
||||
}
|
||||
}
|
||||
};
|
||||
#if __TBB_USE_X86_ELIDING_MUTEX || __TBB_GCC_VERSION < 40000
|
||||
#else
|
||||
public:
|
||||
#endif /* __TBB_USE_X86_ELIDING_MUTEX */
|
||||
|
||||
// Mutex traits
|
||||
static const bool is_rw_mutex = false;
|
||||
static const bool is_recursive_mutex = false;
|
||||
static const bool is_fair_mutex = false;
|
||||
|
||||
// ISO C++0x compatibility methods
|
||||
|
||||
//! Acquire lock
|
||||
void lock() {
|
||||
__TBB_LockByteElided(flag);
|
||||
}
|
||||
|
||||
//! Try acquiring lock (non-blocking)
|
||||
/** Return true if lock acquired; false otherwise. */
|
||||
bool try_lock() {
|
||||
return __TBB_TryLockByteElided(flag);
|
||||
}
|
||||
|
||||
//! Release lock
|
||||
void unlock() {
|
||||
__TBB_UnlockByteElided( flag );
|
||||
}
|
||||
}; // end of x86_eliding_mutex
|
||||
|
||||
} // namespace internal
|
||||
} // namespace interface7
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* ( __TBB_x86_32 || __TBB_x86_64 ) */
|
||||
|
||||
#endif /* __TBB__x86_eliding_mutex_impl_H */
|
||||
@@ -0,0 +1,225 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB__x86_rtm_rw_mutex_impl_H
|
||||
#define __TBB__x86_rtm_rw_mutex_impl_H
|
||||
|
||||
#ifndef __TBB_spin_rw_mutex_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#if __TBB_TSX_AVAILABLE
|
||||
|
||||
#include "../tbb_stddef.h"
|
||||
#include "../tbb_machine.h"
|
||||
#include "../tbb_profiling.h"
|
||||
#include "../spin_rw_mutex.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace interface8 {
|
||||
namespace internal {
|
||||
|
||||
enum RTM_type {
|
||||
RTM_not_in_mutex,
|
||||
RTM_transacting_reader,
|
||||
RTM_transacting_writer,
|
||||
RTM_real_reader,
|
||||
RTM_real_writer
|
||||
};
|
||||
|
||||
static const unsigned long speculation_granularity = 64;
|
||||
|
||||
//! Fast, unfair, spinning speculation-enabled reader-writer lock with backoff and
|
||||
// writer-preference
|
||||
/** @ingroup synchronization */
|
||||
class x86_rtm_rw_mutex: private spin_rw_mutex {
|
||||
#if __TBB_USE_X86_RTM_RW_MUTEX || __TBB_GCC_VERSION < 40000
|
||||
// bug in gcc 3.x.x causes syntax error in spite of the friend declaration below.
|
||||
// Make the scoped_lock public in that case.
|
||||
public:
|
||||
#else
|
||||
private:
|
||||
#endif
|
||||
friend class interface7::internal::padded_mutex<x86_rtm_rw_mutex,true>;
|
||||
class scoped_lock; // should be private
|
||||
friend class scoped_lock;
|
||||
private:
|
||||
//! @cond INTERNAL
|
||||
|
||||
//! Internal construct unacquired mutex.
|
||||
void __TBB_EXPORTED_METHOD internal_construct();
|
||||
|
||||
//! Internal acquire write lock.
|
||||
// only_speculate == true if we're doing a try_lock, else false.
|
||||
void __TBB_EXPORTED_METHOD internal_acquire_writer(x86_rtm_rw_mutex::scoped_lock&, bool only_speculate=false);
|
||||
|
||||
//! Internal acquire read lock.
|
||||
// only_speculate == true if we're doing a try_lock, else false.
|
||||
void __TBB_EXPORTED_METHOD internal_acquire_reader(x86_rtm_rw_mutex::scoped_lock&, bool only_speculate=false);
|
||||
|
||||
//! Internal upgrade reader to become a writer.
|
||||
bool __TBB_EXPORTED_METHOD internal_upgrade( x86_rtm_rw_mutex::scoped_lock& );
|
||||
|
||||
//! Out of line code for downgrading a writer to a reader.
|
||||
bool __TBB_EXPORTED_METHOD internal_downgrade( x86_rtm_rw_mutex::scoped_lock& );
|
||||
|
||||
//! Internal try_acquire write lock.
|
||||
bool __TBB_EXPORTED_METHOD internal_try_acquire_writer( x86_rtm_rw_mutex::scoped_lock& );
|
||||
|
||||
//! Internal release lock.
|
||||
void __TBB_EXPORTED_METHOD internal_release( x86_rtm_rw_mutex::scoped_lock& );
|
||||
|
||||
static x86_rtm_rw_mutex* internal_get_mutex( const spin_rw_mutex::scoped_lock& lock )
|
||||
{
|
||||
return static_cast<x86_rtm_rw_mutex*>( lock.internal_get_mutex() );
|
||||
}
|
||||
static void internal_set_mutex( spin_rw_mutex::scoped_lock& lock, spin_rw_mutex* mtx )
|
||||
{
|
||||
lock.internal_set_mutex( mtx );
|
||||
}
|
||||
//! @endcond
|
||||
public:
|
||||
//! Construct unacquired mutex.
|
||||
x86_rtm_rw_mutex() {
|
||||
w_flag = false;
|
||||
#if TBB_USE_THREADING_TOOLS
|
||||
internal_construct();
|
||||
#endif
|
||||
}
|
||||
|
||||
#if TBB_USE_ASSERT
|
||||
//! Empty destructor.
|
||||
~x86_rtm_rw_mutex() {}
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
|
||||
// Mutex traits
|
||||
static const bool is_rw_mutex = true;
|
||||
static const bool is_recursive_mutex = false;
|
||||
static const bool is_fair_mutex = false;
|
||||
|
||||
#if __TBB_USE_X86_RTM_RW_MUTEX || __TBB_GCC_VERSION < 40000
|
||||
#else
|
||||
// by default we will not provide the scoped_lock interface. The user
|
||||
// should use the padded version of the mutex. scoped_lock is used in
|
||||
// padded_mutex template.
|
||||
private:
|
||||
#endif
|
||||
//! The scoped locking pattern
|
||||
/** It helps to avoid the common problem of forgetting to release lock.
|
||||
It also nicely provides the "node" for queuing locks. */
|
||||
// Speculation-enabled scoped lock for spin_rw_mutex
|
||||
// The idea is to be able to reuse the acquire/release methods of spin_rw_mutex
|
||||
// and its scoped lock wherever possible. The only way to use a speculative lock is to use
|
||||
// a scoped_lock. (because transaction_state must be local)
|
||||
|
||||
class scoped_lock : tbb::internal::no_copy {
|
||||
friend class x86_rtm_rw_mutex;
|
||||
spin_rw_mutex::scoped_lock my_scoped_lock;
|
||||
|
||||
RTM_type transaction_state;
|
||||
|
||||
public:
|
||||
//! Construct lock that has not acquired a mutex.
|
||||
/** Equivalent to zero-initialization of *this. */
|
||||
scoped_lock() : my_scoped_lock(), transaction_state(RTM_not_in_mutex) {
|
||||
}
|
||||
|
||||
//! Acquire lock on given mutex.
|
||||
scoped_lock( x86_rtm_rw_mutex& m, bool write = true ) : my_scoped_lock(),
|
||||
transaction_state(RTM_not_in_mutex) {
|
||||
acquire(m, write);
|
||||
}
|
||||
|
||||
//! Release lock (if lock is held).
|
||||
~scoped_lock() {
|
||||
if(transaction_state != RTM_not_in_mutex) release();
|
||||
}
|
||||
|
||||
//! Acquire lock on given mutex.
|
||||
void acquire( x86_rtm_rw_mutex& m, bool write = true ) {
|
||||
if( write ) m.internal_acquire_writer(*this);
|
||||
else m.internal_acquire_reader(*this);
|
||||
}
|
||||
|
||||
//! Release lock
|
||||
void release() {
|
||||
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
|
||||
__TBB_ASSERT( mutex, "lock is not acquired" );
|
||||
__TBB_ASSERT( transaction_state!=RTM_not_in_mutex, "lock is not acquired" );
|
||||
return mutex->internal_release(*this);
|
||||
}
|
||||
|
||||
//! Upgrade reader to become a writer.
|
||||
/** Returns whether the upgrade happened without releasing and re-acquiring the lock */
|
||||
bool upgrade_to_writer() {
|
||||
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
|
||||
__TBB_ASSERT( mutex, "lock is not acquired" );
|
||||
__TBB_ASSERT( transaction_state==RTM_transacting_reader || transaction_state==RTM_real_reader, "Invalid state for upgrade" );
|
||||
return mutex->internal_upgrade(*this);
|
||||
}
|
||||
|
||||
//! Downgrade writer to become a reader.
|
||||
/** Returns whether the downgrade happened without releasing and re-acquiring the lock */
|
||||
bool downgrade_to_reader() {
|
||||
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
|
||||
__TBB_ASSERT( mutex, "lock is not acquired" );
|
||||
__TBB_ASSERT( transaction_state==RTM_transacting_writer || transaction_state==RTM_real_writer, "Invalid state for downgrade" );
|
||||
return mutex->internal_downgrade(*this);
|
||||
}
|
||||
|
||||
//! Attempt to acquire mutex.
|
||||
/** returns true if successful. */
|
||||
bool try_acquire( x86_rtm_rw_mutex& m, bool write = true ) {
|
||||
#if TBB_USE_ASSERT
|
||||
x86_rtm_rw_mutex* mutex = x86_rtm_rw_mutex::internal_get_mutex(my_scoped_lock);
|
||||
__TBB_ASSERT( !mutex, "lock is already acquired" );
|
||||
#endif
|
||||
// have to assign m to our mutex.
|
||||
// cannot set the mutex, because try_acquire in spin_rw_mutex depends on it being NULL.
|
||||
if(write) return m.internal_try_acquire_writer(*this);
|
||||
// speculatively acquire the lock. If this fails, do try_acquire on the spin_rw_mutex.
|
||||
m.internal_acquire_reader(*this, /*only_speculate=*/true);
|
||||
if(transaction_state == RTM_transacting_reader) return true;
|
||||
if( my_scoped_lock.try_acquire(m, false)) {
|
||||
transaction_state = RTM_real_reader;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
}; // class x86_rtm_rw_mutex::scoped_lock
|
||||
|
||||
// ISO C++0x compatibility methods not provided because we cannot maintain
|
||||
// state about whether a thread is in a transaction.
|
||||
|
||||
private:
|
||||
char pad[speculation_granularity-sizeof(spin_rw_mutex)]; // padding
|
||||
|
||||
// If true, writer holds the spin_rw_mutex.
|
||||
tbb::atomic<bool> w_flag; // want this on a separate cache line
|
||||
|
||||
}; // x86_rtm_rw_mutex
|
||||
|
||||
} // namespace internal
|
||||
} // namespace interface8
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* __TBB_TSX_AVAILABLE */
|
||||
#endif /* __TBB__x86_rtm_rw_mutex_impl_H */
|
||||
@@ -0,0 +1,246 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_intrusive_list_H
|
||||
#define _TBB_intrusive_list_H
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
//! Data structure to be inherited by the types that can form intrusive lists.
|
||||
/** Intrusive list is formed by means of the member_intrusive_list<T> template class.
|
||||
Note that type T must derive from intrusive_list_node either publicly or
|
||||
declare instantiation member_intrusive_list<T> as a friend.
|
||||
This class implements a limited subset of std::list interface. **/
|
||||
struct intrusive_list_node {
|
||||
intrusive_list_node *my_prev_node,
|
||||
*my_next_node;
|
||||
#if TBB_USE_ASSERT
|
||||
intrusive_list_node () { my_prev_node = my_next_node = this; }
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
};
|
||||
|
||||
//! List of element of type T, where T is derived from intrusive_list_node
|
||||
/** The class is not thread safe. **/
|
||||
template <class List, class T>
|
||||
class intrusive_list_base {
|
||||
//! Pointer to the head node
|
||||
intrusive_list_node my_head;
|
||||
|
||||
//! Number of list elements
|
||||
size_t my_size;
|
||||
|
||||
static intrusive_list_node& node ( T& item ) { return List::node(item); }
|
||||
|
||||
static T& item ( intrusive_list_node* node ) { return List::item(node); }
|
||||
|
||||
template<class Iterator>
|
||||
class iterator_impl {
|
||||
Iterator& self () { return *static_cast<Iterator*>(this); }
|
||||
|
||||
//! Node the iterator points to at the moment
|
||||
intrusive_list_node *my_pos;
|
||||
|
||||
protected:
|
||||
iterator_impl (intrusive_list_node* pos )
|
||||
: my_pos(pos)
|
||||
{}
|
||||
|
||||
T& item () const {
|
||||
return intrusive_list_base::item(my_pos);
|
||||
}
|
||||
|
||||
public:
|
||||
iterator_impl () : my_pos(NULL) {}
|
||||
|
||||
Iterator& operator = ( const Iterator& it ) {
|
||||
return my_pos = it.my_pos;
|
||||
}
|
||||
|
||||
Iterator& operator = ( const T& val ) {
|
||||
return my_pos = &node(val);
|
||||
}
|
||||
|
||||
bool operator == ( const Iterator& it ) const {
|
||||
return my_pos == it.my_pos;
|
||||
}
|
||||
|
||||
bool operator != ( const Iterator& it ) const {
|
||||
return my_pos != it.my_pos;
|
||||
}
|
||||
|
||||
Iterator& operator++ () {
|
||||
my_pos = my_pos->my_next_node;
|
||||
return self();
|
||||
}
|
||||
|
||||
Iterator& operator-- () {
|
||||
my_pos = my_pos->my_prev_node;
|
||||
return self();
|
||||
}
|
||||
|
||||
Iterator operator++ ( int ) {
|
||||
Iterator result = self();
|
||||
++(*this);
|
||||
return result;
|
||||
}
|
||||
|
||||
Iterator operator-- ( int ) {
|
||||
Iterator result = self();
|
||||
--(*this);
|
||||
return result;
|
||||
}
|
||||
}; // intrusive_list_base::iterator_impl
|
||||
|
||||
void assert_ok () const {
|
||||
__TBB_ASSERT( (my_head.my_prev_node == &my_head && !my_size) ||
|
||||
(my_head.my_next_node != &my_head && my_size >0), "intrusive_list_base corrupted" );
|
||||
#if TBB_USE_ASSERT >= 2
|
||||
size_t i = 0;
|
||||
for ( intrusive_list_node *n = my_head.my_next_node; n != &my_head; n = n->my_next_node )
|
||||
++i;
|
||||
__TBB_ASSERT( my_size == i, "Wrong size" );
|
||||
#endif /* TBB_USE_ASSERT >= 2 */
|
||||
}
|
||||
|
||||
public:
|
||||
class iterator : public iterator_impl<iterator> {
|
||||
template <class U, class V> friend class intrusive_list_base;
|
||||
public:
|
||||
iterator (intrusive_list_node* pos )
|
||||
: iterator_impl<iterator>(pos )
|
||||
{}
|
||||
iterator () {}
|
||||
|
||||
T* operator-> () const { return &this->item(); }
|
||||
|
||||
T& operator* () const { return this->item(); }
|
||||
}; // class iterator
|
||||
|
||||
class const_iterator : public iterator_impl<const_iterator> {
|
||||
template <class U, class V> friend class intrusive_list_base;
|
||||
public:
|
||||
const_iterator (const intrusive_list_node* pos )
|
||||
: iterator_impl<const_iterator>(const_cast<intrusive_list_node*>(pos) )
|
||||
{}
|
||||
const_iterator () {}
|
||||
|
||||
const T* operator-> () const { return &this->item(); }
|
||||
|
||||
const T& operator* () const { return this->item(); }
|
||||
}; // class iterator
|
||||
|
||||
intrusive_list_base () : my_size(0) {
|
||||
my_head.my_prev_node = &my_head;
|
||||
my_head.my_next_node = &my_head;
|
||||
}
|
||||
|
||||
bool empty () const { return my_head.my_next_node == &my_head; }
|
||||
|
||||
size_t size () const { return my_size; }
|
||||
|
||||
iterator begin () { return iterator(my_head.my_next_node); }
|
||||
|
||||
iterator end () { return iterator(&my_head); }
|
||||
|
||||
const_iterator begin () const { return const_iterator(my_head.my_next_node); }
|
||||
|
||||
const_iterator end () const { return const_iterator(&my_head); }
|
||||
|
||||
void push_front ( T& val ) {
|
||||
__TBB_ASSERT( node(val).my_prev_node == &node(val) && node(val).my_next_node == &node(val),
|
||||
"Object with intrusive list node can be part of only one intrusive list simultaneously" );
|
||||
// An object can be part of only one intrusive list at the given moment via the given node member
|
||||
node(val).my_prev_node = &my_head;
|
||||
node(val).my_next_node = my_head.my_next_node;
|
||||
my_head.my_next_node->my_prev_node = &node(val);
|
||||
my_head.my_next_node = &node(val);
|
||||
++my_size;
|
||||
assert_ok();
|
||||
}
|
||||
|
||||
void remove( T& val ) {
|
||||
__TBB_ASSERT( node(val).my_prev_node != &node(val) && node(val).my_next_node != &node(val), "Element to remove is not in the list" );
|
||||
__TBB_ASSERT( node(val).my_prev_node->my_next_node == &node(val) && node(val).my_next_node->my_prev_node == &node(val), "Element to remove is not in the list" );
|
||||
--my_size;
|
||||
node(val).my_next_node->my_prev_node = node(val).my_prev_node;
|
||||
node(val).my_prev_node->my_next_node = node(val).my_next_node;
|
||||
#if TBB_USE_ASSERT
|
||||
node(val).my_prev_node = node(val).my_next_node = &node(val);
|
||||
#endif
|
||||
assert_ok();
|
||||
}
|
||||
|
||||
iterator erase ( iterator it ) {
|
||||
T& val = *it;
|
||||
++it;
|
||||
remove( val );
|
||||
return it;
|
||||
}
|
||||
|
||||
}; // intrusive_list_base
|
||||
|
||||
|
||||
//! Double linked list of items of type T containing a member of type intrusive_list_node.
|
||||
/** NodePtr is a member pointer to the node data field. Class U is either T or
|
||||
a base class of T containing the node member. Default values exist for the sake
|
||||
of a partial specialization working with inheritance case.
|
||||
|
||||
The list does not have ownership of its items. Its purpose is to avoid dynamic
|
||||
memory allocation when forming lists of existing objects.
|
||||
|
||||
The class is not thread safe. **/
|
||||
template <class T, class U, intrusive_list_node U::*NodePtr>
|
||||
class memptr_intrusive_list : public intrusive_list_base<memptr_intrusive_list<T, U, NodePtr>, T>
|
||||
{
|
||||
friend class intrusive_list_base<memptr_intrusive_list<T, U, NodePtr>, T>;
|
||||
|
||||
static intrusive_list_node& node ( T& val ) { return val.*NodePtr; }
|
||||
|
||||
static T& item ( intrusive_list_node* node ) {
|
||||
// Cannot use __TBB_offsetof (and consequently __TBB_get_object_ref) macro
|
||||
// with *NodePtr argument because gcc refuses to interpret pasted "->" and "*"
|
||||
// as member pointer dereferencing operator, and explicit usage of ## in
|
||||
// __TBB_offsetof implementation breaks operations with normal member names.
|
||||
return *reinterpret_cast<T*>((char*)node - ((ptrdiff_t)&(reinterpret_cast<T*>(0x1000)->*NodePtr) - 0x1000));
|
||||
}
|
||||
}; // intrusive_list<T, U, NodePtr>
|
||||
|
||||
//! Double linked list of items of type T that is derived from intrusive_list_node class.
|
||||
/** The list does not have ownership of its items. Its purpose is to avoid dynamic
|
||||
memory allocation when forming lists of existing objects.
|
||||
|
||||
The class is not thread safe. **/
|
||||
template <class T>
|
||||
class intrusive_list : public intrusive_list_base<intrusive_list<T>, T>
|
||||
{
|
||||
friend class intrusive_list_base<intrusive_list<T>, T>;
|
||||
|
||||
static intrusive_list_node& node ( T& val ) { return val; }
|
||||
|
||||
static T& item ( intrusive_list_node* node ) { return *static_cast<T*>(node); }
|
||||
}; // intrusive_list<T>
|
||||
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#endif /* _TBB_intrusive_list_H */
|
||||
@@ -0,0 +1,92 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if DO_ITT_NOTIFY
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#ifndef UNICODE
|
||||
#define UNICODE
|
||||
#endif
|
||||
#else
|
||||
#pragma weak dlopen
|
||||
#pragma weak dlsym
|
||||
#pragma weak dlerror
|
||||
#endif /* WIN */
|
||||
|
||||
#if __TBB_BUILD
|
||||
|
||||
extern "C" void ITT_DoOneTimeInitialization();
|
||||
#define __itt_init_ittlib_name(x,y) (ITT_DoOneTimeInitialization(), true)
|
||||
|
||||
#elif __TBBMALLOC_BUILD
|
||||
|
||||
extern "C" void MallocInitializeITT();
|
||||
#define __itt_init_ittlib_name(x,y) (MallocInitializeITT(), true)
|
||||
|
||||
#else
|
||||
#error This file is expected to be used for either TBB or TBB allocator build.
|
||||
#endif // __TBB_BUILD
|
||||
|
||||
#include "tools_api/ittnotify_static.c"
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
int __TBB_load_ittnotify() {
|
||||
return __itt_init_ittlib(NULL, // groups for:
|
||||
(__itt_group_id)(__itt_group_sync // prepare/cancel/acquired/releasing
|
||||
| __itt_group_thread // name threads
|
||||
| __itt_group_stitch // stack stitching
|
||||
#if __TBB_CPF_BUILD
|
||||
| __itt_group_structure
|
||||
#endif
|
||||
));
|
||||
}
|
||||
|
||||
}} // namespaces
|
||||
|
||||
#endif /* DO_ITT_NOTIFY */
|
||||
|
||||
#define __TBB_NO_IMPLICIT_LINKAGE 1
|
||||
#include "itt_notify.h"
|
||||
|
||||
namespace tbb {
|
||||
|
||||
#if DO_ITT_NOTIFY
|
||||
const tchar
|
||||
*SyncType_GlobalLock = _T("TbbGlobalLock"),
|
||||
*SyncType_Scheduler = _T("%Constant")
|
||||
;
|
||||
const tchar
|
||||
*SyncObj_SchedulerInitialization = _T("TbbSchedulerInitialization"),
|
||||
*SyncObj_SchedulersList = _T("TbbSchedulersList"),
|
||||
*SyncObj_WorkerLifeCycleMgmt = _T("TBB Scheduler"),
|
||||
*SyncObj_TaskStealingLoop = _T("TBB Scheduler"),
|
||||
*SyncObj_WorkerTaskPool = _T("TBB Scheduler"),
|
||||
*SyncObj_MasterTaskPool = _T("TBB Scheduler"),
|
||||
*SyncObj_TaskPoolSpinning = _T("TBB Scheduler"),
|
||||
*SyncObj_Mailbox = _T("TBB Scheduler"),
|
||||
*SyncObj_TaskReturnList = _T("TBB Scheduler"),
|
||||
*SyncObj_TaskStream = _T("TBB Scheduler"),
|
||||
*SyncObj_ContextsList = _T("TBB Scheduler")
|
||||
;
|
||||
#endif /* DO_ITT_NOTIFY */
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
128
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/itt_notify.h
vendored
Normal file
128
Plugins/GameLiftPlugin/Source/GameLiftServer/ThirdParty/concurrentqueue/benchmarks/tbb/itt_notify.h
vendored
Normal file
@@ -0,0 +1,128 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef _TBB_ITT_NOTIFY
|
||||
#define _TBB_ITT_NOTIFY
|
||||
|
||||
#include "tbb/tbb_stddef.h"
|
||||
|
||||
#if DO_ITT_NOTIFY
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#ifndef UNICODE
|
||||
#define UNICODE
|
||||
#endif
|
||||
#endif /* WIN */
|
||||
|
||||
#ifndef INTEL_ITTNOTIFY_API_PRIVATE
|
||||
#define INTEL_ITTNOTIFY_API_PRIVATE
|
||||
#endif
|
||||
|
||||
#include "tools_api/ittnotify.h"
|
||||
#include "tools_api/legacy/ittnotify.h"
|
||||
extern "C" void __itt_fini_ittlib(void);
|
||||
|
||||
#if _WIN32||_WIN64
|
||||
#undef _T
|
||||
#undef __itt_event_create
|
||||
#define __itt_event_create __itt_event_createA
|
||||
#endif /* WIN */
|
||||
|
||||
|
||||
#endif /* DO_ITT_NOTIFY */
|
||||
|
||||
#if !ITT_CALLER_NULL
|
||||
#define ITT_CALLER_NULL ((__itt_caller)0)
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
//! Unicode support
|
||||
#if (_WIN32||_WIN64) && !__MINGW32__
|
||||
//! Unicode character type. Always wchar_t on Windows.
|
||||
/** We do not use typedefs from Windows TCHAR family to keep consistence of TBB coding style. **/
|
||||
typedef wchar_t tchar;
|
||||
//! Standard Windows macro to markup the string literals.
|
||||
#define _T(string_literal) L ## string_literal
|
||||
#else /* !WIN */
|
||||
typedef char tchar;
|
||||
//! Standard Windows style macro to markup the string literals.
|
||||
#define _T(string_literal) string_literal
|
||||
#endif /* !WIN */
|
||||
} // namespace tbb
|
||||
|
||||
#if DO_ITT_NOTIFY
|
||||
namespace tbb {
|
||||
//! Display names of internal synchronization types
|
||||
extern const tchar
|
||||
*SyncType_GlobalLock,
|
||||
*SyncType_Scheduler;
|
||||
//! Display names of internal synchronization components/scenarios
|
||||
extern const tchar
|
||||
*SyncObj_SchedulerInitialization,
|
||||
*SyncObj_SchedulersList,
|
||||
*SyncObj_WorkerLifeCycleMgmt,
|
||||
*SyncObj_TaskStealingLoop,
|
||||
*SyncObj_WorkerTaskPool,
|
||||
*SyncObj_MasterTaskPool,
|
||||
*SyncObj_TaskPoolSpinning,
|
||||
*SyncObj_Mailbox,
|
||||
*SyncObj_TaskReturnList,
|
||||
*SyncObj_TaskStream,
|
||||
*SyncObj_ContextsList
|
||||
;
|
||||
|
||||
namespace internal {
|
||||
void __TBB_EXPORTED_FUNC itt_set_sync_name_v3( void* obj, const tchar* name);
|
||||
|
||||
} // namespace internal
|
||||
|
||||
} // namespace tbb
|
||||
|
||||
// const_cast<void*>() is necessary to cast off volatility
|
||||
#define ITT_NOTIFY(name,obj) __itt_notify_##name(const_cast<void*>(static_cast<volatile void*>(obj)))
|
||||
#define ITT_THREAD_SET_NAME(name) __itt_thread_set_name(name)
|
||||
#define ITT_FINI_ITTLIB() __itt_fini_ittlib()
|
||||
#define ITT_SYNC_CREATE(obj, type, name) __itt_sync_create((void*)(obj), type, name, 2)
|
||||
#define ITT_SYNC_RENAME(obj, name) __itt_sync_rename(obj, name)
|
||||
#define ITT_STACK_CREATE(obj) obj = __itt_stack_caller_create()
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
#define ITT_STACK(precond, name, obj) (precond) ? __itt_stack_##name(obj) : ((void)0);
|
||||
#else
|
||||
#define ITT_STACK(precond, name, obj) ((void)0)
|
||||
#endif /* !__TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
#else /* !DO_ITT_NOTIFY */
|
||||
|
||||
#define ITT_NOTIFY(name,obj) ((void)0)
|
||||
#define ITT_THREAD_SET_NAME(name) ((void)0)
|
||||
#define ITT_FINI_ITTLIB() ((void)0)
|
||||
#define ITT_SYNC_CREATE(obj, type, name) ((void)0)
|
||||
#define ITT_SYNC_RENAME(obj, name) ((void)0)
|
||||
#define ITT_STACK_CREATE(obj) ((void)0)
|
||||
#define ITT_STACK(precond, name, obj) ((void)0)
|
||||
|
||||
#endif /* !DO_ITT_NOTIFY */
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
int __TBB_load_ittnotify();
|
||||
}}
|
||||
|
||||
#endif /* _TBB_ITT_NOTIFY */
|
||||
@@ -0,0 +1,49 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
{
|
||||
global:
|
||||
|
||||
#define __TBB_SYMBOL( sym ) sym;
|
||||
#include "lin32-tbb-export.lst"
|
||||
|
||||
local:
|
||||
|
||||
/* TBB symbols */
|
||||
*3tbb*;
|
||||
*__TBB*;
|
||||
|
||||
/* ITT symbols */
|
||||
__itt_*;
|
||||
|
||||
/* Intel Compiler (libirc) symbols */
|
||||
__intel_*;
|
||||
_intel_*;
|
||||
get_memcpy_largest_cachelinesize;
|
||||
get_memcpy_largest_cache_size;
|
||||
get_mem_ops_method;
|
||||
init_mem_ops_method;
|
||||
irc__get_msg;
|
||||
irc__print;
|
||||
override_mem_ops_method;
|
||||
set_memcpy_largest_cachelinesize;
|
||||
set_memcpy_largest_cache_size;
|
||||
|
||||
};
|
||||
@@ -0,0 +1,391 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/* cache_aligned_allocator.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEjjPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Ej )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
|
||||
|
||||
/* task.cpp v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
|
||||
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEj )
|
||||
__TBB_SYMBOL( _ZTIN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEij )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
/* arena.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEi )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* task_v2.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
|
||||
#endif /* !TBB_NO_LEGACY */
|
||||
|
||||
/* Exception handling in task scheduler */
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
/* Symbols for exceptions thrown from TBB */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
|
||||
|
||||
/* tbb_misc.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
|
||||
#if __TBB_x86_32
|
||||
__TBB_SYMBOL( __TBB_machine_store8_slow_perf_warning )
|
||||
__TBB_SYMBOL( __TBB_machine_store8_slow )
|
||||
#endif
|
||||
__TBB_SYMBOL( TBB_runtime_interface_version )
|
||||
|
||||
/* tbb_main.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
|
||||
#if __TBB_ITT_STRUCTURE_API
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22itt_make_task_group_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23itt_metadata_str_add_v7ENS0_15itt_domain_enumEPvyNS0_12string_indexEPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_relation_add_v7ENS0_15itt_domain_enumEPvyNS0_12itt_relationES2_y )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17itt_task_begin_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15itt_task_end_v7ENS0_15itt_domain_enumE )
|
||||
#endif
|
||||
|
||||
/* pipeline.cpp */
|
||||
__TBB_SYMBOL( _ZTIN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEj )
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEjRNS_18task_group_contextE )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
|
||||
|
||||
/* queuing_rw_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
|
||||
|
||||
/* reader_writer_lock.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* spin_rw_mutex.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
|
||||
#endif
|
||||
|
||||
/* spin_rw_mutex v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
|
||||
|
||||
// x86_rtm_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
|
||||
|
||||
/* spin_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
|
||||
/* mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
|
||||
|
||||
/* recursive_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
|
||||
|
||||
/* QueuingMutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
|
||||
|
||||
/* critical_section.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_hash_map */
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
|
||||
|
||||
/* concurrent_queue.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityEij )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Ej )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_queue v3 */
|
||||
/* constructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Ej )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Ej )
|
||||
/* destructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
|
||||
/* typeinfo */
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* vtable */
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* methods */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityEij )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_vector.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_jPFvPvPKvjE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvjEb )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_jPFvPvjEPFvS4_PKvjESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEjjPFvPvjE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEjjj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEjRj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEjjPFvPvjE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_vector v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_jPFvPvPKvjE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvjE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_jPFvPvjEPFvS4_PKvjESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEjjPFvPvPKvjES4_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEjjj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEjRj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEjjPFvPvPKvjES4_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEjPvPFvS2_jEPFvS2_PKvjE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEjjjPKvPFvPvjEPFvS4_S3_jE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEjjPFvPvPKvjES4_ )
|
||||
|
||||
/* tbb_thread */
|
||||
#if __MINGW32__
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFjPvES2_ )
|
||||
#else
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Ej )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
|
||||
|
||||
#if __MINGW32__
|
||||
/* condition_variable */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal32internal_condition_variable_waitERNS1_14condvar_impl_tEPNS_5mutexEPKNS_10tick_count10interval_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal35internal_destroy_condition_variableERNS1_14condvar_impl_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_condition_variable_notify_allERNS1_14condvar_impl_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_condition_variable_notify_oneERNS1_14condvar_impl_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal38internal_initialize_condition_variableERNS1_14condvar_impl_tE )
|
||||
#endif
|
||||
|
||||
#undef __TBB_SYMBOL
|
||||
@@ -0,0 +1,46 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
{
|
||||
global:
|
||||
|
||||
#define __TBB_SYMBOL( sym ) sym;
|
||||
#include "lin64-tbb-export.lst"
|
||||
|
||||
local:
|
||||
|
||||
/* TBB symbols */
|
||||
*3tbb*;
|
||||
*__TBB*;
|
||||
|
||||
/* ITT symbols */
|
||||
__itt_*;
|
||||
|
||||
/* Intel Compiler (libirc) symbols */
|
||||
__intel_*;
|
||||
_intel_*;
|
||||
get_msg_buf;
|
||||
get_text_buf;
|
||||
message_catalog;
|
||||
print_buf;
|
||||
irc__get_msg;
|
||||
irc__print;
|
||||
|
||||
};
|
||||
@@ -0,0 +1,373 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/* cache_aligned_allocator.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
|
||||
|
||||
/* task.cpp v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
|
||||
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZTIN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
/* arena.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* task_v2.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
|
||||
#endif /* !TBB_NO_LEGACY */
|
||||
|
||||
/* Exception handling in task scheduler */
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
/* Symbols for exceptions thrown from TBB */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
|
||||
/* tbb_misc.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
|
||||
__TBB_SYMBOL( TBB_runtime_interface_version )
|
||||
|
||||
/* tbb_main.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
|
||||
#if __TBB_ITT_STRUCTURE_API
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23itt_metadata_str_add_v7ENS0_15itt_domain_enumEPvyNS0_12string_indexEPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22itt_make_task_group_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17itt_task_begin_v7ENS0_15itt_domain_enumEPvyS2_yNS0_12string_indexE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_relation_add_v7ENS0_15itt_domain_enumEPvyNS0_12itt_relationES2_y )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15itt_task_end_v7ENS0_15itt_domain_enumE )
|
||||
#endif
|
||||
|
||||
/* pipeline.cpp */
|
||||
__TBB_SYMBOL( _ZTIN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
|
||||
|
||||
/* queuing_rw_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
|
||||
|
||||
/* reader_writer_lock.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* spin_rw_mutex.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
|
||||
#endif
|
||||
|
||||
// x86_rtm_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
|
||||
|
||||
/* spin_rw_mutex v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
|
||||
|
||||
/* spin_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
|
||||
|
||||
/* mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
|
||||
|
||||
/* recursive_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
|
||||
|
||||
/* QueuingMutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
|
||||
|
||||
/* critical_section.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_hash_map */
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
|
||||
|
||||
/* concurrent_queue.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_queue v3 */
|
||||
/* constructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
|
||||
/* destructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
|
||||
/* typeinfo */
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* vtable */
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* methods */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_vector.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_vector v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
|
||||
|
||||
/* tbb_thread */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
|
||||
|
||||
#undef __TBB_SYMBOL
|
||||
@@ -0,0 +1,48 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
{
|
||||
global:
|
||||
|
||||
#define __TBB_SYMBOL( sym ) sym;
|
||||
#include "lin64ipf-tbb-export.lst"
|
||||
|
||||
local:
|
||||
|
||||
/* TBB symbols */
|
||||
*3tbb*;
|
||||
*__TBB*;
|
||||
|
||||
/* ITT symbols */
|
||||
__itt_*;
|
||||
|
||||
/* Intel Compiler (libirc) symbols */
|
||||
__intel_*;
|
||||
_intel_*;
|
||||
?0_memcopyA;
|
||||
?0_memcopyDu;
|
||||
?0_memcpyD;
|
||||
?1__memcpy;
|
||||
?1__memmove;
|
||||
?1__serial_memmove;
|
||||
memcpy;
|
||||
memset;
|
||||
|
||||
};
|
||||
@@ -0,0 +1,408 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/* cache_aligned_allocator.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
|
||||
|
||||
/* task.cpp v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
|
||||
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZTIN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
/* arena.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* task_v2.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
|
||||
#endif /* !TBB_NO_LEGACY */
|
||||
|
||||
/* Exception handling in task scheduler */
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZN3tbb13tbb_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
/* Symbols for exceptions thrown from TBB */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
|
||||
|
||||
/* tbb_misc.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
|
||||
__TBB_SYMBOL( TBB_runtime_interface_version )
|
||||
|
||||
/* tbb_main.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
|
||||
|
||||
/* pipeline.cpp */
|
||||
__TBB_SYMBOL( _ZTIN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
|
||||
|
||||
/* queuing_rw_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
|
||||
|
||||
/* reader_writer_lock.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* spin_rw_mutex.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
|
||||
#endif
|
||||
|
||||
/* spin_rw_mutex v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
|
||||
|
||||
/* spin_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
|
||||
/* mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
|
||||
|
||||
/* recursive_mutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
|
||||
|
||||
/* QueuingMutex.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
|
||||
|
||||
/* critical_section.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_hash_map */
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
|
||||
|
||||
/* concurrent_queue.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_queue v3 */
|
||||
/* constructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
|
||||
/* destructors */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
|
||||
/* typeinfo */
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* vtable */
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
|
||||
/* methods */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
/* concurrent_vector.cpp v2 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
|
||||
#endif
|
||||
|
||||
/* concurrent_vector v3 */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
|
||||
|
||||
/* tbb_thread */
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
|
||||
|
||||
/* asm functions */
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd1__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd2__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd4__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd8__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore1__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore2__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore4__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore8__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd1acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd1release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd2acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd2release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd4acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd4release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd8acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchadd8release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore1acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore1release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore2acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore2release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore4acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore4release )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore8acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_fetchstore8release )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp1acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp1release )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp1__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp2acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp2release )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp2__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp4acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp4release )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp4__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp8acquire )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp8release )
|
||||
__TBB_SYMBOL( __TBB_machine_cmpswp8__TBB_full_fence )
|
||||
__TBB_SYMBOL( __TBB_machine_lg )
|
||||
__TBB_SYMBOL( __TBB_machine_lockbyte )
|
||||
__TBB_SYMBOL( __TBB_machine_pause )
|
||||
__TBB_SYMBOL( __TBB_machine_trylockbyte )
|
||||
__TBB_SYMBOL( __TBB_machine_load8_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_store8_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_load4_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_store4_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_load2_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_store2_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_load1_relaxed )
|
||||
__TBB_SYMBOL( __TBB_machine_store1_relaxed )
|
||||
|
||||
#undef __TBB_SYMBOL
|
||||
@@ -0,0 +1,23 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#define __TBB_SYMBOL( sym ) _##sym
|
||||
#include "mac32-tbb-export.lst"
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/*
|
||||
Sometimes OS X* requires leading underscore (e. g. in export list file), but sometimes not
|
||||
(e. g. when searching symbol in a dynamic library via dlsym()). Symbols in this file SHOULD
|
||||
be listed WITHOUT one leading underscore. __TBB_SYMBOL macro should add underscore when
|
||||
necessary, depending on the indended usage.
|
||||
*/
|
||||
|
||||
// cache_aligned_allocator.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
|
||||
|
||||
// task.cpp v3
|
||||
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
|
||||
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZTIN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
/* arena.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// task_v2.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
|
||||
#endif
|
||||
|
||||
// Exception handling in task scheduler
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
// Symbols for exceptions thrown from TBB
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
|
||||
__TBB_SYMBOL( _ZNSt13runtime_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt13runtime_error )
|
||||
__TBB_SYMBOL( _ZTSSt13runtime_error )
|
||||
__TBB_SYMBOL( _ZNSt16invalid_argumentD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt16invalid_argument )
|
||||
__TBB_SYMBOL( _ZTSSt16invalid_argument )
|
||||
__TBB_SYMBOL( _ZNSt11range_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt11range_error )
|
||||
__TBB_SYMBOL( _ZTSSt11range_error )
|
||||
__TBB_SYMBOL( _ZNSt12length_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt12length_error )
|
||||
__TBB_SYMBOL( _ZTSSt12length_error )
|
||||
__TBB_SYMBOL( _ZNSt12out_of_rangeD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt12out_of_range )
|
||||
__TBB_SYMBOL( _ZTSSt12out_of_range )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
|
||||
|
||||
// tbb_misc.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
|
||||
#if __TBB_x86_32
|
||||
__TBB_SYMBOL( __TBB_machine_store8_slow_perf_warning )
|
||||
__TBB_SYMBOL( __TBB_machine_store8_slow )
|
||||
#endif
|
||||
__TBB_SYMBOL( TBB_runtime_interface_version )
|
||||
|
||||
// tbb_main.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
|
||||
|
||||
// pipeline.cpp
|
||||
__TBB_SYMBOL( _ZTIN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
|
||||
|
||||
// queuing_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
|
||||
|
||||
// reader_writer_lock.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// spin_rw_mutex.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
|
||||
#endif
|
||||
|
||||
// spin_rw_mutex v3
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
|
||||
|
||||
// x86_rtm_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
|
||||
|
||||
// spin_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
|
||||
|
||||
// mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
|
||||
|
||||
// recursive_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
|
||||
|
||||
// queuing_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
|
||||
|
||||
// critical_section.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// concurrent_hash_map
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
|
||||
|
||||
// concurrent_queue.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityEim )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
|
||||
#endif
|
||||
|
||||
// concurrent_queue v3
|
||||
// constructors
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
|
||||
// destructors
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
|
||||
// typeinfo
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
|
||||
// vtable
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
|
||||
// methods
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityEim )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// concurrent_vector.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
|
||||
#endif
|
||||
|
||||
// concurrent_vector v3
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
|
||||
|
||||
// tbb_thread
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
|
||||
|
||||
#undef __TBB_SYMBOL
|
||||
@@ -0,0 +1,23 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#define __TBB_SYMBOL( sym ) _##sym
|
||||
#include "mac64-tbb-export.lst"
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#include "tbb/tbb_config.h"
|
||||
|
||||
/*
|
||||
Sometimes OS X* requires leading underscore (e. g. in export list file), but sometimes not
|
||||
(e. g. when searching symbol in a dynamic library via dlsym()). Symbols in this file SHOULD
|
||||
be listed WITHOUT one leading underscore. __TBB_SYMBOL macro should add underscore when
|
||||
necessary, depending on the indended usage.
|
||||
*/
|
||||
|
||||
// cache_aligned_allocator.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal12NFS_AllocateEmmPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15NFS_GetLineSizeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal8NFS_FreeEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal23allocate_via_handler_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25deallocate_via_handler_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal17is_malloc_used_v3Ev )
|
||||
|
||||
// task.cpp v3
|
||||
__TBB_SYMBOL( _ZN3tbb4task13note_affinityEt )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22internal_set_ref_countEi )
|
||||
__TBB_SYMBOL( _ZN3tbb4task28internal_decrement_ref_countEv )
|
||||
__TBB_SYMBOL( _ZN3tbb4task22spawn_and_wait_for_allERNS_9task_listE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task4selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface58internal9task_base7destroyERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb4task26is_owned_by_current_threadEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_root_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal28affinity_partitioner_base_v36resizeEj )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal36get_initial_auto_partitioner_divisorEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal20allocate_child_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal27allocate_continuation_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal34allocate_additional_child_of_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZTIN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init19default_num_threadsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEim )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init10initializeEi )
|
||||
__TBB_SYMBOL( _ZN3tbb19task_scheduler_init9terminateEv )
|
||||
#if __TBB_SCHEDULER_OBSERVER
|
||||
__TBB_SYMBOL( _ZN3tbb8internal26task_scheduler_observer_v37observeEb )
|
||||
#endif /* __TBB_SCHEDULER_OBSERVER */
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_task7executeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10empty_taskD1Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10empty_taskE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10empty_taskE )
|
||||
|
||||
#if __TBB_TASK_ARENA
|
||||
/* arena.cpp */
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base18internal_terminateEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_enqueueERNS_4taskEl )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base16internal_executeERNS1_13delegate_baseE )
|
||||
__TBB_SYMBOL( _ZNK3tbb10interface78internal15task_arena_base13internal_waitEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface78internal15task_arena_base21internal_current_slotEv )
|
||||
#endif /* __TBB_TASK_ARENA */
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// task_v2.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb4task7destroyERS0_ )
|
||||
#endif
|
||||
|
||||
// Exception handling in task scheduler
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy8allocateEm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal32allocate_root_with_context_proxy4freeERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb4task12change_groupERNS_18task_group_contextE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context28is_group_execution_cancelledEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context22cancel_group_executionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context26register_pending_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context5resetEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context19capture_fp_settingsEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context4initEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_contextD2Ev )
|
||||
#if __TBB_TASK_PRIORITY
|
||||
__TBB_SYMBOL( _ZN3tbb18task_group_context12set_priorityENS_10priority_tE )
|
||||
__TBB_SYMBOL( _ZNK3tbb18task_group_context8priorityEv )
|
||||
#endif /* __TBB_TASK_PRIORITY */
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4nameEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb18captured_exception4whatEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception10throw_selfEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception3setEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception4moveEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception7destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exception8allocateEPKcS2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb18captured_exceptionD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb18captured_exceptionE )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13tbb_exceptionE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13tbb_exceptionE )
|
||||
#endif /* __TBB_TASK_GROUP_CONTEXT */
|
||||
|
||||
// Symbols for exceptions thrown from TBB
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33throw_bad_last_alloc_exception_v4Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18throw_exception_v4ENS0_12exception_idE )
|
||||
__TBB_SYMBOL( _ZNSt13runtime_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt13runtime_error )
|
||||
__TBB_SYMBOL( _ZTSSt13runtime_error )
|
||||
__TBB_SYMBOL( _ZNSt16invalid_argumentD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt16invalid_argument )
|
||||
__TBB_SYMBOL( _ZTSSt16invalid_argument )
|
||||
__TBB_SYMBOL( _ZNSt11range_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt11range_error )
|
||||
__TBB_SYMBOL( _ZTSSt11range_error )
|
||||
__TBB_SYMBOL( _ZNSt12length_errorD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt12length_error )
|
||||
__TBB_SYMBOL( _ZTSSt12length_error )
|
||||
__TBB_SYMBOL( _ZNSt12out_of_rangeD1Ev )
|
||||
__TBB_SYMBOL( _ZTISt12out_of_range )
|
||||
__TBB_SYMBOL( _ZTSSt12out_of_range )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb14bad_last_allocD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb14bad_last_alloc4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb14bad_last_allocE )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb12missing_waitD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb12missing_wait4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb12missing_waitE )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb27invalid_multiple_schedulingD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb27invalid_multiple_scheduling4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb27invalid_multiple_schedulingE )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb13improper_lockD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb13improper_lock4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb13improper_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb10user_abortD1Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb10user_abort4whatEv )
|
||||
__TBB_SYMBOL( _ZTIN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb10user_abortE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb10user_abortE )
|
||||
|
||||
|
||||
// tbb_misc.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb17assertion_failureEPKciS1_S1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb21set_assertion_handlerEPFvPKciS1_S1_E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13handle_perrorEiPKc )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15runtime_warningEPKcz )
|
||||
__TBB_SYMBOL( TBB_runtime_interface_version )
|
||||
|
||||
// tbb_main.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal32itt_load_pointer_with_acquire_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33itt_store_pointer_with_release_v3EPvS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal18call_itt_notify_v5EiPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19itt_load_pointer_v3EPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal20itt_set_sync_name_v3EPvPKc )
|
||||
|
||||
// pipeline.cpp
|
||||
__TBB_SYMBOL( _ZTIN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filterD2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline10add_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline12inject_tokenERNS_4taskE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline13remove_filterERNS_6filterE )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEm )
|
||||
#if __TBB_TASK_GROUP_CONTEXT
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline3runEmRNS_18task_group_contextE )
|
||||
#endif
|
||||
__TBB_SYMBOL( _ZN3tbb8pipeline5clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter12process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb19thread_bound_filter16try_process_itemEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineC2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD0Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD1Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8pipelineD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8pipelineE )
|
||||
__TBB_SYMBOL( _ZN3tbb6filter16set_end_of_inputEv )
|
||||
|
||||
// queuing_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock17upgrade_to_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock19downgrade_to_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex11scoped_lock11try_acquireERS0_b )
|
||||
__TBB_SYMBOL( _ZN3tbb16queuing_rw_mutex18internal_constructEv )
|
||||
|
||||
// reader_writer_lock.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock11scoped_lock18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock13try_lock_readEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16scoped_lock_read18internal_constructERS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock4lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock6unlockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock8try_lockEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface518reader_writer_lock9lock_readEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// spin_rw_mutex.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex16internal_upgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex22internal_itt_releasingEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_acquire_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex18internal_downgradeEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex23internal_release_writerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_readerEPS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13spin_rw_mutex27internal_try_acquire_writerEPS0_ )
|
||||
#endif
|
||||
|
||||
// spin_rw_mutex v3
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v316internal_upgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_downgradeEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v323internal_release_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_readerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v327internal_try_acquire_writerEv )
|
||||
__TBB_SYMBOL( _ZN3tbb16spin_rw_mutex_v318internal_constructEv )
|
||||
|
||||
// x86_rtm_rw_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_releaseERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex16internal_upgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_constructEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex18internal_downgradeERNS2_11scoped_lockE )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_readerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex23internal_acquire_writerERNS2_11scoped_lockEb )
|
||||
__TBB_SYMBOL( _ZN3tbb10interface88internal16x86_rtm_rw_mutex27internal_try_acquire_writerERNS2_11scoped_lockE )
|
||||
|
||||
// spin_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb10spin_mutex18internal_constructEv )
|
||||
|
||||
// mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb5mutex18internal_constructEv )
|
||||
|
||||
// recursive_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock16internal_releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex11scoped_lock20internal_try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex16internal_destroyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb15recursive_mutex18internal_constructEv )
|
||||
|
||||
// queuing_mutex.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock7releaseEv )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex11scoped_lock11try_acquireERS0_ )
|
||||
__TBB_SYMBOL( _ZN3tbb13queuing_mutex18internal_constructEv )
|
||||
|
||||
// critical_section.cpp
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19critical_section_v418internal_constructEv )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// concurrent_hash_map
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21hash_map_segment_base23internal_grow_predicateEv )
|
||||
|
||||
// concurrent_queue.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base12internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base13internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base21internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base23internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_base25internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseC2Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal21concurrent_queue_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base6assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_base7advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseC2ERKNS0_21concurrent_queue_baseE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal30concurrent_queue_iterator_baseD2Ev )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal21concurrent_queue_base13internal_sizeEv )
|
||||
#endif
|
||||
|
||||
// concurrent_queue v3
|
||||
// constructors
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3C2ERKNS0_24concurrent_queue_base_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3C2Em )
|
||||
// destructors
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v3D2Ev )
|
||||
// typeinfo
|
||||
__TBB_SYMBOL( _ZTIN3tbb8internal24concurrent_queue_base_v3E )
|
||||
__TBB_SYMBOL( _ZTSN3tbb8internal24concurrent_queue_base_v3E )
|
||||
// vtable
|
||||
__TBB_SYMBOL( _ZTVN3tbb8internal24concurrent_queue_base_v3E )
|
||||
// methods
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal33concurrent_queue_iterator_base_v37advanceEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v313internal_pushEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v818internal_push_moveEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v325internal_push_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v830internal_push_move_if_not_fullEPKv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v312internal_popEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v323internal_pop_if_presentEPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v314internal_abortEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_finish_clearEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v321internal_set_capacityElm )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v313internal_sizeEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v314internal_emptyEv )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal24concurrent_queue_base_v324internal_throw_exceptionEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v36assignERKS1_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal24concurrent_queue_base_v812move_contentERS1_ )
|
||||
|
||||
#if !TBB_NO_LEGACY
|
||||
// concurrent_vector.cpp v2
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base13internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base14internal_clearEPFvPvmEb )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base15internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_grow_byEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base16internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base18internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal22concurrent_vector_base25internal_grow_to_at_leastEmmPFvPvmE )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal22concurrent_vector_base17internal_capacityEv )
|
||||
#endif
|
||||
|
||||
// concurrent_vector v3
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_copyERKS1_mPFvPvPKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v314internal_clearEPFvPvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_assignERKS1_mPFvPvmEPFvS4_PKvmESA_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_grow_byEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_reserveEmmm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v318internal_push_backEmRm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v325internal_grow_to_at_leastEmmPFvPvPKvmES4_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v317internal_capacityEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v316internal_compactEmPvPFvS2_mEPFvS2_PKvmE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v313internal_swapERS1_ )
|
||||
__TBB_SYMBOL( _ZNK3tbb8internal25concurrent_vector_base_v324internal_throw_exceptionEm )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v3D2Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v315internal_resizeEmmmPKvPFvPvmEPFvS4_S3_mE )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal25concurrent_vector_base_v337internal_grow_to_at_least_with_resultEmmPFvPvPKvmES4_ )
|
||||
|
||||
// tbb_thread
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v320hardware_concurrencyEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v36detachEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal16thread_get_id_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15free_closure_v3EPv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v34joinEv )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal13tbb_thread_v314internal_startEPFPvS2_ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal19allocate_closure_v3Em )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal7move_v3ERNS0_13tbb_thread_v3ES2_ )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_yield_v3Ev )
|
||||
__TBB_SYMBOL( _ZN3tbb8internal15thread_sleep_v3ERKNS_10tick_count10interval_tE )
|
||||
|
||||
#undef __TBB_SYMBOL
|
||||
@@ -0,0 +1,217 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
/*
|
||||
Platform isolation layer for the ARMv7-a architecture.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_machine_H
|
||||
#error Do not include this file directly; include tbb_machine.h instead
|
||||
#endif
|
||||
|
||||
//TODO: is ARMv7 is the only version ever to support?
|
||||
#if !(__ARM_ARCH_7A__)
|
||||
#error compilation requires an ARMv7-a architecture.
|
||||
#endif
|
||||
|
||||
#include <sys/param.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#define __TBB_WORDSIZE 4
|
||||
|
||||
// Traditionally ARM is little-endian.
|
||||
// Note that, since only the layout of aligned 32-bit words is of interest,
|
||||
// any apparent PDP-endianness of 32-bit words at half-word alignment or
|
||||
// any little-endian ordering of big-endian 32-bit words in 64-bit quantities
|
||||
// may be disregarded for this setting.
|
||||
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
|
||||
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
#elif defined(__BYTE_ORDER__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
|
||||
#else
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
|
||||
#endif
|
||||
|
||||
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("dmb ish": : :"memory")
|
||||
#define __TBB_control_consistency_helper() __TBB_full_memory_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_full_memory_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_full_memory_fence()
|
||||
|
||||
//--------------------------------------------------
|
||||
// Compare and swap
|
||||
//--------------------------------------------------
|
||||
|
||||
/**
|
||||
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
|
||||
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
|
||||
* @param value value to assign *ptr to if *ptr==comparand
|
||||
* @param comparand value to compare with *ptr
|
||||
* @return value originally in memory at ptr, regardless of success
|
||||
*/
|
||||
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand )
|
||||
{
|
||||
int32_t oldval, res;
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
do {
|
||||
__asm__ __volatile__(
|
||||
"ldrex %1, [%3]\n"
|
||||
"mov %0, #0\n"
|
||||
"cmp %1, %4\n"
|
||||
"it eq\n"
|
||||
"strexeq %0, %5, [%3]\n"
|
||||
: "=&r" (res), "=&r" (oldval), "+Qo" (*(volatile int32_t*)ptr)
|
||||
: "r" ((int32_t *)ptr), "Ir" (comparand), "r" (value)
|
||||
: "cc");
|
||||
} while (res);
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
return oldval;
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomic CAS for 64 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
|
||||
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
|
||||
* @param value value to assign *ptr to if *ptr==comparand
|
||||
* @param comparand value to compare with *ptr
|
||||
* @return value originally in memory at ptr, regardless of success
|
||||
*/
|
||||
static inline int64_t __TBB_machine_cmpswp8(volatile void *ptr, int64_t value, int64_t comparand )
|
||||
{
|
||||
int64_t oldval;
|
||||
int32_t res;
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
do {
|
||||
__asm__ __volatile__(
|
||||
"mov %0, #0\n"
|
||||
"ldrexd %1, %H1, [%3]\n"
|
||||
"cmp %1, %4\n"
|
||||
"it eq\n"
|
||||
"cmpeq %H1, %H4\n"
|
||||
"it eq\n"
|
||||
"strexdeq %0, %5, %H5, [%3]"
|
||||
: "=&r" (res), "=&r" (oldval), "+Qo" (*(volatile int64_t*)ptr)
|
||||
: "r" ((int64_t *)ptr), "r" (comparand), "r" (value)
|
||||
: "cc");
|
||||
} while (res);
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
return oldval;
|
||||
}
|
||||
|
||||
static inline int32_t __TBB_machine_fetchadd4(volatile void* ptr, int32_t addend)
|
||||
{
|
||||
unsigned long tmp;
|
||||
int32_t result, tmp2;
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
__asm__ __volatile__(
|
||||
"1: ldrex %0, [%4]\n"
|
||||
" add %3, %0, %5\n"
|
||||
" strex %1, %3, [%4]\n"
|
||||
" cmp %1, #0\n"
|
||||
" bne 1b\n"
|
||||
: "=&r" (result), "=&r" (tmp), "+Qo" (*(volatile int32_t*)ptr), "=&r"(tmp2)
|
||||
: "r" ((int32_t *)ptr), "Ir" (addend)
|
||||
: "cc");
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend)
|
||||
{
|
||||
unsigned long tmp;
|
||||
int64_t result, tmp2;
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
__asm__ __volatile__(
|
||||
"1: ldrexd %0, %H0, [%4]\n"
|
||||
" adds %3, %0, %5\n"
|
||||
" adc %H3, %H0, %H5\n"
|
||||
" strexd %1, %3, %H3, [%4]\n"
|
||||
" cmp %1, #0\n"
|
||||
" bne 1b"
|
||||
: "=&r" (result), "=&r" (tmp), "+Qo" (*(volatile int64_t*)ptr), "=&r"(tmp2)
|
||||
: "r" ((int64_t *)ptr), "r" (addend)
|
||||
: "cc");
|
||||
|
||||
|
||||
__TBB_full_memory_fence();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
inline void __TBB_machine_pause (int32_t delay )
|
||||
{
|
||||
while(delay>0)
|
||||
{
|
||||
__TBB_compiler_fence();
|
||||
delay--;
|
||||
}
|
||||
}
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
template <typename T, size_t S>
|
||||
struct machine_load_store_relaxed {
|
||||
static inline T load ( const volatile T& location ) {
|
||||
const T value = location;
|
||||
|
||||
/*
|
||||
* An extra memory barrier is required for errata #761319
|
||||
* Please see http://infocenter.arm.com/help/topic/com.arm.doc.uan0004a
|
||||
*/
|
||||
__TBB_acquire_consistency_helper();
|
||||
return value;
|
||||
}
|
||||
|
||||
static inline void store ( volatile T& location, T value ) {
|
||||
location = value;
|
||||
}
|
||||
};
|
||||
}} // namespaces internal, tbb
|
||||
|
||||
// Machine specific atomic operations
|
||||
|
||||
#define __TBB_CompareAndSwap4(P,V,C) __TBB_machine_cmpswp4(P,V,C)
|
||||
#define __TBB_CompareAndSwap8(P,V,C) __TBB_machine_cmpswp8(P,V,C)
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
|
||||
// Use generics for some things
|
||||
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
@@ -0,0 +1,131 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_generic_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_gcc_generic_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#define __TBB_WORDSIZE __SIZEOF_POINTER__
|
||||
|
||||
#if __TBB_GCC_64BIT_ATOMIC_BUILTINS_BROKEN
|
||||
#define __TBB_64BIT_ATOMICS 0
|
||||
#endif
|
||||
|
||||
/** FPU control setting not available for non-Intel architectures on Android **/
|
||||
#if __ANDROID__ && __TBB_generic_arch
|
||||
#define __TBB_CPU_CTL_ENV_PRESENT 0
|
||||
#endif
|
||||
|
||||
// __BYTE_ORDER__ is used in accordance with http://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html,
|
||||
// but __BIG_ENDIAN__ or __LITTLE_ENDIAN__ may be more commonly found instead.
|
||||
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
|
||||
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
#elif defined(__BYTE_ORDER__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
|
||||
#else
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
|
||||
#endif
|
||||
|
||||
/** As this generic implementation has absolutely no information about underlying
|
||||
hardware, its performance most likely will be sub-optimal because of full memory
|
||||
fence usages where a more lightweight synchronization means (or none at all)
|
||||
could suffice. Thus if you use this header to enable TBB on a new platform,
|
||||
consider forking it and relaxing below helpers as appropriate. **/
|
||||
#define __TBB_acquire_consistency_helper() __sync_synchronize()
|
||||
#define __TBB_release_consistency_helper() __sync_synchronize()
|
||||
#define __TBB_full_memory_fence() __sync_synchronize()
|
||||
#define __TBB_control_consistency_helper() __sync_synchronize()
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T) \
|
||||
inline T __TBB_machine_cmpswp##S( volatile void *ptr, T value, T comparand ) { \
|
||||
return __sync_val_compare_and_swap(reinterpret_cast<volatile T *>(ptr),comparand,value); \
|
||||
} \
|
||||
\
|
||||
inline T __TBB_machine_fetchadd##S( volatile void *ptr, T value ) { \
|
||||
return __sync_fetch_and_add(reinterpret_cast<volatile T *>(ptr),value); \
|
||||
} \
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t)
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_ATOMICS
|
||||
|
||||
namespace tbb{ namespace internal { namespace gcc_builtins {
|
||||
inline int clz(unsigned int x){ return __builtin_clz(x);}
|
||||
inline int clz(unsigned long int x){ return __builtin_clzl(x);}
|
||||
inline int clz(unsigned long long int x){ return __builtin_clzll(x);}
|
||||
}}}
|
||||
//gcc __builtin_clz builtin count _number_ of leading zeroes
|
||||
static inline intptr_t __TBB_machine_lg( uintptr_t x ) {
|
||||
return sizeof(x)*8 - tbb::internal::gcc_builtins::clz(x) -1 ;
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_or( volatile void *ptr, uintptr_t addend ) {
|
||||
__sync_fetch_and_or(reinterpret_cast<volatile uintptr_t *>(ptr),addend);
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_and( volatile void *ptr, uintptr_t addend ) {
|
||||
__sync_fetch_and_and(reinterpret_cast<volatile uintptr_t *>(ptr),addend);
|
||||
}
|
||||
|
||||
|
||||
typedef unsigned char __TBB_Flag;
|
||||
|
||||
typedef __TBB_atomic __TBB_Flag __TBB_atomic_flag;
|
||||
|
||||
inline bool __TBB_machine_try_lock_byte( __TBB_atomic_flag &flag ) {
|
||||
return __sync_lock_test_and_set(&flag,1)==0;
|
||||
}
|
||||
|
||||
inline void __TBB_machine_unlock_byte( __TBB_atomic_flag &flag ) {
|
||||
__sync_lock_release(&flag);
|
||||
}
|
||||
|
||||
// Machine specific atomic operations
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
|
||||
|
||||
#define __TBB_TryLockByte __TBB_machine_try_lock_byte
|
||||
#define __TBB_UnlockByte __TBB_machine_unlock_byte
|
||||
|
||||
// Definition of other functions
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#if __TBB_WORDSIZE==4
|
||||
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
|
||||
#endif
|
||||
|
||||
#if __TBB_x86_32 || __TBB_x86_64
|
||||
#include "gcc_itsx.h"
|
||||
#endif
|
||||
@@ -0,0 +1,100 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_machine_gcc_ia32_common_H
|
||||
#define __TBB_machine_gcc_ia32_common_H
|
||||
|
||||
//TODO: Add a higher-level function, e.g. tbb::interal::log2(), into tbb_stddef.h, which
|
||||
//uses __TBB_Log2 and contains the assert and remove the assert from here and all other
|
||||
//platform-specific headers.
|
||||
//TODO: Check if use of gcc intrinsic gives a better chance for cross call optimizations
|
||||
template <typename T>
|
||||
static inline intptr_t __TBB_machine_lg( T x ) {
|
||||
__TBB_ASSERT(x>0, "The logarithm of a non-positive value is undefined.");
|
||||
uintptr_t j;
|
||||
__asm__("bsr %1,%0" : "=r"(j) : "r"((uintptr_t)x));
|
||||
return j;
|
||||
}
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
|
||||
#ifndef __TBB_Pause
|
||||
//TODO: check if raising a ratio of pause instructions to loop control instructions
|
||||
//(via e.g. loop unrolling) gives any benefit for HT. E.g, the current implementation
|
||||
//does about 2 CPU-consuming instructions for every pause instruction. Perhaps for
|
||||
//high pause counts it should use an unrolled loop to raise the ratio, and thus free
|
||||
//up more integer cycles for the other hyperthread. On the other hand, if the loop is
|
||||
//unrolled too far, it won't fit in the core's loop cache, and thus take away
|
||||
//instruction decode slots from the other hyperthread.
|
||||
|
||||
//TODO: check if use of gcc __builtin_ia32_pause intrinsic gives a "some how" better performing code
|
||||
static inline void __TBB_machine_pause( int32_t delay ) {
|
||||
for (int32_t i = 0; i < delay; i++) {
|
||||
__asm__ __volatile__("pause;");
|
||||
}
|
||||
return;
|
||||
}
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
#endif /* !__TBB_Pause */
|
||||
|
||||
// API to retrieve/update FPU control setting
|
||||
#ifndef __TBB_CPU_CTL_ENV_PRESENT
|
||||
#define __TBB_CPU_CTL_ENV_PRESENT 1
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
class cpu_ctl_env {
|
||||
private:
|
||||
int mxcsr;
|
||||
short x87cw;
|
||||
static const int MXCSR_CONTROL_MASK = ~0x3f; /* all except last six status bits */
|
||||
public:
|
||||
bool operator!=( const cpu_ctl_env& ctl ) const { return mxcsr != ctl.mxcsr || x87cw != ctl.x87cw; }
|
||||
void get_env() {
|
||||
#if __TBB_ICC_12_0_INL_ASM_FSTCW_BROKEN
|
||||
cpu_ctl_env loc_ctl;
|
||||
__asm__ __volatile__ (
|
||||
"stmxcsr %0\n\t"
|
||||
"fstcw %1"
|
||||
: "=m"(loc_ctl.mxcsr), "=m"(loc_ctl.x87cw)
|
||||
);
|
||||
*this = loc_ctl;
|
||||
#else
|
||||
__asm__ __volatile__ (
|
||||
"stmxcsr %0\n\t"
|
||||
"fstcw %1"
|
||||
: "=m"(mxcsr), "=m"(x87cw)
|
||||
);
|
||||
#endif
|
||||
mxcsr &= MXCSR_CONTROL_MASK;
|
||||
}
|
||||
void set_env() const {
|
||||
__asm__ __volatile__ (
|
||||
"ldmxcsr %0\n\t"
|
||||
"fldcw %1"
|
||||
: : "m"(mxcsr), "m"(x87cw)
|
||||
);
|
||||
}
|
||||
};
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
#endif /* !__TBB_CPU_CTL_ENV_PRESENT */
|
||||
|
||||
#include "gcc_itsx.h"
|
||||
|
||||
#endif /* __TBB_machine_gcc_ia32_common_H */
|
||||
@@ -0,0 +1,123 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_itsx_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_gcc_itsx_H
|
||||
|
||||
#define __TBB_OP_XACQUIRE 0xF2
|
||||
#define __TBB_OP_XRELEASE 0xF3
|
||||
#define __TBB_OP_LOCK 0xF0
|
||||
|
||||
#define __TBB_STRINGIZE_INTERNAL(arg) #arg
|
||||
#define __TBB_STRINGIZE(arg) __TBB_STRINGIZE_INTERNAL(arg)
|
||||
|
||||
#ifdef __TBB_x86_64
|
||||
#define __TBB_r_out "=r"
|
||||
#else
|
||||
#define __TBB_r_out "=q"
|
||||
#endif
|
||||
|
||||
inline static uint8_t __TBB_machine_try_lock_elided( volatile uint8_t* lk )
|
||||
{
|
||||
uint8_t value = 1;
|
||||
__asm__ volatile (".byte " __TBB_STRINGIZE(__TBB_OP_XACQUIRE)"; lock; xchgb %0, %1;"
|
||||
: __TBB_r_out(value), "=m"(*lk) : "0"(value), "m"(*lk) : "memory" );
|
||||
return uint8_t(value^1);
|
||||
}
|
||||
|
||||
inline static void __TBB_machine_try_lock_elided_cancel()
|
||||
{
|
||||
// 'pause' instruction aborts HLE/RTM transactions
|
||||
__asm__ volatile ("pause\n" : : : "memory" );
|
||||
}
|
||||
|
||||
inline static void __TBB_machine_unlock_elided( volatile uint8_t* lk )
|
||||
{
|
||||
__asm__ volatile (".byte " __TBB_STRINGIZE(__TBB_OP_XRELEASE)"; movb $0, %0"
|
||||
: "=m"(*lk) : "m"(*lk) : "memory" );
|
||||
}
|
||||
|
||||
#if __TBB_TSX_INTRINSICS_PRESENT
|
||||
#include <immintrin.h>
|
||||
|
||||
#define __TBB_machine_is_in_transaction _xtest
|
||||
#define __TBB_machine_begin_transaction _xbegin
|
||||
#define __TBB_machine_end_transaction _xend
|
||||
#define __TBB_machine_transaction_conflict_abort() _xabort(0xff)
|
||||
|
||||
#else
|
||||
|
||||
/*!
|
||||
* Check if the instruction is executed in a transaction or not
|
||||
*/
|
||||
inline static bool __TBB_machine_is_in_transaction()
|
||||
{
|
||||
int8_t res = 0;
|
||||
#if __TBB_x86_32
|
||||
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD6;\n"
|
||||
"setz %0" : "=q"(res) : : "memory" );
|
||||
#else
|
||||
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD6;\n"
|
||||
"setz %0" : "=r"(res) : : "memory" );
|
||||
#endif
|
||||
return res==0;
|
||||
}
|
||||
|
||||
/*!
|
||||
* Enter speculative execution mode.
|
||||
* @return -1 on success
|
||||
* abort cause ( or 0 ) on abort
|
||||
*/
|
||||
inline static uint32_t __TBB_machine_begin_transaction()
|
||||
{
|
||||
uint32_t res = ~uint32_t(0); // success value
|
||||
__asm__ volatile ("1: .byte 0xC7; .byte 0xF8;\n" // XBEGIN <abort-offset>
|
||||
" .long 2f-1b-6\n" // 2f-1b == difference in addresses of start
|
||||
// of XBEGIN and the MOVL
|
||||
// 2f - 1b - 6 == that difference minus the size of the
|
||||
// XBEGIN instruction. This is the abort offset to
|
||||
// 2: below.
|
||||
" jmp 3f\n" // success (leave -1 in res)
|
||||
"2: movl %%eax,%0\n" // store failure code in res
|
||||
"3:"
|
||||
:"=r"(res):"0"(res):"memory","%eax");
|
||||
return res;
|
||||
}
|
||||
|
||||
/*!
|
||||
* Attempt to commit/end transaction
|
||||
*/
|
||||
inline static void __TBB_machine_end_transaction()
|
||||
{
|
||||
__asm__ volatile (".byte 0x0F; .byte 0x01; .byte 0xD5" :::"memory"); // XEND
|
||||
}
|
||||
|
||||
/*
|
||||
* aborts with code 0xFF (lock already held)
|
||||
*/
|
||||
inline static void __TBB_machine_transaction_conflict_abort()
|
||||
{
|
||||
__asm__ volatile (".byte 0xC6; .byte 0xF8; .byte 0xFF" :::"memory");
|
||||
}
|
||||
|
||||
#endif /* __TBB_TSX_INTRINSICS_PRESENT */
|
||||
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
// TODO: revise by comparing with mac_ppc.h
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_ibm_aix51_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_ibm_aix51_H
|
||||
|
||||
#define __TBB_WORDSIZE 8
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG // assumption based on operating system
|
||||
|
||||
#include <stdint.h>
|
||||
#include <unistd.h>
|
||||
#include <sched.h>
|
||||
|
||||
extern "C" {
|
||||
int32_t __TBB_machine_cas_32 (volatile void* ptr, int32_t value, int32_t comparand);
|
||||
int64_t __TBB_machine_cas_64 (volatile void* ptr, int64_t value, int64_t comparand);
|
||||
void __TBB_machine_flush ();
|
||||
void __TBB_machine_lwsync ();
|
||||
void __TBB_machine_isync ();
|
||||
}
|
||||
|
||||
// Mapping of old entry point names retained for the sake of backward binary compatibility
|
||||
#define __TBB_machine_cmpswp4 __TBB_machine_cas_32
|
||||
#define __TBB_machine_cmpswp8 __TBB_machine_cas_64
|
||||
|
||||
#define __TBB_Yield() sched_yield()
|
||||
|
||||
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
|
||||
#define __TBB_USE_GENERIC_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#if __GNUC__
|
||||
#define __TBB_control_consistency_helper() __asm__ __volatile__( "isync": : :"memory")
|
||||
#define __TBB_acquire_consistency_helper() __asm__ __volatile__("lwsync": : :"memory")
|
||||
#define __TBB_release_consistency_helper() __asm__ __volatile__("lwsync": : :"memory")
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__( "sync": : :"memory")
|
||||
#else
|
||||
// IBM C++ Compiler does not support inline assembly
|
||||
// TODO: Since XL 9.0 or earlier GCC syntax is supported. Replace with more
|
||||
// lightweight implementation (like in mac_ppc.h)
|
||||
#define __TBB_control_consistency_helper() __TBB_machine_isync ()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_machine_lwsync ()
|
||||
#define __TBB_release_consistency_helper() __TBB_machine_lwsync ()
|
||||
#define __TBB_full_memory_fence() __TBB_machine_flush ()
|
||||
#endif
|
||||
@@ -0,0 +1,258 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_icc_generic_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#if ! __TBB_ICC_BUILTIN_ATOMICS_PRESENT
|
||||
#error "Intel C++ Compiler of at least 12.0 version is needed to use ICC intrinsics port"
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_icc_generic_H
|
||||
|
||||
//ICC mimics the "native" target compiler
|
||||
#if _MSC_VER
|
||||
#include "msvc_ia32_common.h"
|
||||
#else
|
||||
#include "gcc_ia32_common.h"
|
||||
#endif
|
||||
|
||||
//TODO: Make __TBB_WORDSIZE macro optional for ICC intrinsics port.
|
||||
//As compiler intrinsics are used for all the operations it is possible to do.
|
||||
|
||||
#if __TBB_x86_32
|
||||
#define __TBB_WORDSIZE 4
|
||||
#else
|
||||
#define __TBB_WORDSIZE 8
|
||||
#endif
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
|
||||
//__TBB_compiler_fence() defined just in case, as it seems not to be used on its own anywhere else
|
||||
#if _MSC_VER
|
||||
//TODO: any way to use same intrinsics on windows and linux?
|
||||
#pragma intrinsic(_ReadWriteBarrier)
|
||||
#define __TBB_compiler_fence() _ReadWriteBarrier()
|
||||
#else
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
|
||||
#endif
|
||||
|
||||
#ifndef __TBB_full_memory_fence
|
||||
#if _MSC_VER
|
||||
//TODO: any way to use same intrinsics on windows and linux?
|
||||
#pragma intrinsic(_mm_mfence)
|
||||
#define __TBB_full_memory_fence() _mm_mfence()
|
||||
#else
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
|
||||
namespace tbb { namespace internal {
|
||||
//TODO: is there any way to reuse definition of memory_order enum from ICC instead of copy paste.
|
||||
//however it seems unlikely that ICC will silently change exact enum values, as they are defined
|
||||
//in the ISO exactly like this.
|
||||
//TODO: add test that exact values of the enum are same as in the ISO C++11
|
||||
typedef enum memory_order {
|
||||
memory_order_relaxed, memory_order_consume, memory_order_acquire,
|
||||
memory_order_release, memory_order_acq_rel, memory_order_seq_cst
|
||||
} memory_order;
|
||||
|
||||
namespace icc_intrinsics_port {
|
||||
template <typename T>
|
||||
T convert_argument(T value){
|
||||
return value;
|
||||
}
|
||||
//The overload below is needed to have explicit conversion of pointer to void* in argument list.
|
||||
//compiler bug?
|
||||
//TODO: add according broken macro and recheck with ICC 13.0 if the overload is still needed
|
||||
template <typename T>
|
||||
void* convert_argument(T* value){
|
||||
return (void*)value;
|
||||
}
|
||||
}
|
||||
//TODO: code below is a bit repetitive, consider simplifying it
|
||||
template <typename T, size_t S>
|
||||
struct machine_load_store {
|
||||
static T load_with_acquire ( const volatile T& location ) {
|
||||
return __atomic_load_explicit(&location, memory_order_acquire);
|
||||
}
|
||||
static void store_with_release ( volatile T &location, T value ) {
|
||||
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_release);
|
||||
}
|
||||
};
|
||||
|
||||
template <typename T, size_t S>
|
||||
struct machine_load_store_relaxed {
|
||||
static inline T load ( const T& location ) {
|
||||
return __atomic_load_explicit(&location, memory_order_relaxed);
|
||||
}
|
||||
static inline void store ( T& location, T value ) {
|
||||
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_relaxed);
|
||||
}
|
||||
};
|
||||
|
||||
template <typename T, size_t S>
|
||||
struct machine_load_store_seq_cst {
|
||||
static T load ( const volatile T& location ) {
|
||||
return __atomic_load_explicit(&location, memory_order_seq_cst);
|
||||
}
|
||||
|
||||
static void store ( volatile T &location, T value ) {
|
||||
__atomic_store_explicit(&location, value, memory_order_seq_cst);
|
||||
}
|
||||
};
|
||||
|
||||
}} // namespace tbb::internal
|
||||
|
||||
namespace tbb{ namespace internal { namespace icc_intrinsics_port{
|
||||
typedef enum memory_order_map {
|
||||
relaxed = memory_order_relaxed,
|
||||
acquire = memory_order_acquire,
|
||||
release = memory_order_release,
|
||||
full_fence= memory_order_seq_cst
|
||||
} memory_order_map;
|
||||
}}}// namespace tbb::internal
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,M) \
|
||||
inline T __TBB_machine_cmpswp##S##M( volatile void *ptr, T value, T comparand ) { \
|
||||
__atomic_compare_exchange_strong_explicit( \
|
||||
(T*)ptr \
|
||||
,&comparand \
|
||||
,value \
|
||||
, tbb::internal::icc_intrinsics_port::M \
|
||||
, tbb::internal::icc_intrinsics_port::M); \
|
||||
return comparand; \
|
||||
} \
|
||||
\
|
||||
inline T __TBB_machine_fetchstore##S##M(volatile void *ptr, T value) { \
|
||||
return __atomic_exchange_explicit((T*)ptr, value, tbb::internal::icc_intrinsics_port::M); \
|
||||
} \
|
||||
\
|
||||
inline T __TBB_machine_fetchadd##S##M(volatile void *ptr, T value) { \
|
||||
return __atomic_fetch_add_explicit((T*)ptr, value, tbb::internal::icc_intrinsics_port::M); \
|
||||
} \
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, full_fence)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, acquire)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, release)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,tbb::internal::int8_t, relaxed)
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, full_fence)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, acquire)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, release)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,tbb::internal::int16_t, relaxed)
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, full_fence)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, acquire)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, release)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,tbb::internal::int32_t, relaxed)
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, full_fence)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, acquire)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, release)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,tbb::internal::int64_t, relaxed)
|
||||
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_ATOMICS
|
||||
|
||||
#define __TBB_USE_FENCED_ATOMICS 1
|
||||
|
||||
namespace tbb { namespace internal {
|
||||
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
|
||||
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(full_fence)
|
||||
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(full_fence)
|
||||
|
||||
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(acquire)
|
||||
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(release)
|
||||
|
||||
__TBB_MACHINE_DEFINE_LOAD8_GENERIC_FENCED(relaxed)
|
||||
__TBB_MACHINE_DEFINE_STORE8_GENERIC_FENCED(relaxed)
|
||||
|
||||
template <typename T>
|
||||
struct machine_load_store<T,8> {
|
||||
static T load_with_acquire ( const volatile T& location ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
return __atomic_load_explicit(&location, memory_order_acquire);
|
||||
} else {
|
||||
return __TBB_machine_generic_load8acquire(&location);
|
||||
}
|
||||
}
|
||||
static void store_with_release ( volatile T &location, T value ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_release);
|
||||
} else {
|
||||
return __TBB_machine_generic_store8release(&location,value);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
template <typename T>
|
||||
struct machine_load_store_relaxed<T,8> {
|
||||
static T load( const volatile T& location ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
return __atomic_load_explicit(&location, memory_order_relaxed);
|
||||
} else {
|
||||
return __TBB_machine_generic_load8relaxed(&location);
|
||||
}
|
||||
}
|
||||
static void store( volatile T &location, T value ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
__atomic_store_explicit(&location, icc_intrinsics_port::convert_argument(value), memory_order_relaxed);
|
||||
} else {
|
||||
return __TBB_machine_generic_store8relaxed(&location,value);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
template <typename T >
|
||||
struct machine_load_store_seq_cst<T,8> {
|
||||
static T load ( const volatile T& location ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
return __atomic_load_explicit(&location, memory_order_seq_cst);
|
||||
} else {
|
||||
return __TBB_machine_generic_load8full_fence(&location);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
static void store ( volatile T &location, T value ) {
|
||||
if( tbb::internal::is_aligned(&location,8)) {
|
||||
__atomic_store_explicit(&location, value, memory_order_seq_cst);
|
||||
} else {
|
||||
return __TBB_machine_generic_store8full_fence(&location,value);
|
||||
}
|
||||
|
||||
}
|
||||
};
|
||||
|
||||
#endif
|
||||
}} // namespace tbb::internal
|
||||
template <typename T>
|
||||
inline void __TBB_machine_OR( T *operand, T addend ) {
|
||||
__atomic_fetch_or_explicit(operand, addend, tbb::internal::memory_order_seq_cst);
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline void __TBB_machine_AND( T *operand, T addend ) {
|
||||
__atomic_fetch_and_explicit(operand, addend, tbb::internal::memory_order_seq_cst);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_machine_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#include <sched.h>
|
||||
#define __TBB_Yield() sched_yield()
|
||||
|
||||
#include <unistd.h>
|
||||
/* Futex definitions */
|
||||
#include <sys/syscall.h>
|
||||
|
||||
#if defined(SYS_futex)
|
||||
|
||||
#define __TBB_USE_FUTEX 1
|
||||
#include <limits.h>
|
||||
#include <errno.h>
|
||||
// Unfortunately, some versions of Linux do not have a header that defines FUTEX_WAIT and FUTEX_WAKE.
|
||||
|
||||
#ifdef FUTEX_WAIT
|
||||
#define __TBB_FUTEX_WAIT FUTEX_WAIT
|
||||
#else
|
||||
#define __TBB_FUTEX_WAIT 0
|
||||
#endif
|
||||
|
||||
#ifdef FUTEX_WAKE
|
||||
#define __TBB_FUTEX_WAKE FUTEX_WAKE
|
||||
#else
|
||||
#define __TBB_FUTEX_WAKE 1
|
||||
#endif
|
||||
|
||||
#ifndef __TBB_ASSERT
|
||||
#error machine specific headers must be included after tbb_stddef.h
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
|
||||
namespace internal {
|
||||
|
||||
inline int futex_wait( void *futex, int comparand ) {
|
||||
int r = syscall( SYS_futex,futex,__TBB_FUTEX_WAIT,comparand,NULL,NULL,0 );
|
||||
#if TBB_USE_ASSERT
|
||||
int e = errno;
|
||||
__TBB_ASSERT( r==0||r==EWOULDBLOCK||(r==-1&&(e==EAGAIN||e==EINTR)), "futex_wait failed." );
|
||||
#endif /* TBB_USE_ASSERT */
|
||||
return r;
|
||||
}
|
||||
|
||||
inline int futex_wakeup_one( void *futex ) {
|
||||
int r = ::syscall( SYS_futex,futex,__TBB_FUTEX_WAKE,1,NULL,NULL,0 );
|
||||
__TBB_ASSERT( r==0||r==1, "futex_wakeup_one: more than one thread woken up?" );
|
||||
return r;
|
||||
}
|
||||
|
||||
inline int futex_wakeup_all( void *futex ) {
|
||||
int r = ::syscall( SYS_futex,futex,__TBB_FUTEX_WAKE,INT_MAX,NULL,NULL,0 );
|
||||
__TBB_ASSERT( r>=0, "futex_wakeup_all: error in waking up threads" );
|
||||
return r;
|
||||
}
|
||||
|
||||
} /* namespace internal */
|
||||
|
||||
} /* namespace tbb */
|
||||
|
||||
#endif /* SYS_futex */
|
||||
@@ -0,0 +1,232 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_ia32_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_linux_ia32_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include "gcc_ia32_common.h"
|
||||
|
||||
#define __TBB_WORDSIZE 4
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
|
||||
|
||||
#if __TBB_ICC_ASM_VOLATILE_BROKEN
|
||||
#define __TBB_VOLATILE
|
||||
#else
|
||||
#define __TBB_VOLATILE volatile
|
||||
#endif
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,X,R) \
|
||||
static inline T __TBB_machine_cmpswp##S (volatile void *ptr, T value, T comparand ) \
|
||||
{ \
|
||||
T result; \
|
||||
\
|
||||
__asm__ __volatile__("lock\ncmpxchg" X " %2,%1" \
|
||||
: "=a"(result), "=m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "q"(value), "0"(comparand), "m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
\
|
||||
static inline T __TBB_machine_fetchadd##S(volatile void *ptr, T addend) \
|
||||
{ \
|
||||
T result; \
|
||||
__asm__ __volatile__("lock\nxadd" X " %0,%1" \
|
||||
: R (result), "=m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "0"(addend), "m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
\
|
||||
static inline T __TBB_machine_fetchstore##S(volatile void *ptr, T value) \
|
||||
{ \
|
||||
T result; \
|
||||
__asm__ __volatile__("lock\nxchg" X " %0,%1" \
|
||||
: R (result), "=m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "0"(value), "m"(*(__TBB_VOLATILE T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t,"","=q")
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t,"","=r")
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t,"l","=r")
|
||||
|
||||
#if __INTEL_COMPILER
|
||||
#pragma warning( push )
|
||||
// reference to EBX in a function requiring stack alignment
|
||||
#pragma warning( disable: 998 )
|
||||
#endif
|
||||
|
||||
#if __TBB_GCC_CAS8_BUILTIN_INLINING_BROKEN
|
||||
#define __TBB_IA32_CAS8_NOINLINE __attribute__ ((noinline))
|
||||
#else
|
||||
#define __TBB_IA32_CAS8_NOINLINE
|
||||
#endif
|
||||
|
||||
static inline __TBB_IA32_CAS8_NOINLINE int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand ) {
|
||||
//TODO: remove the extra part of condition once __TBB_GCC_BUILTIN_ATOMICS_PRESENT is lowered to gcc version 4.1.2
|
||||
#if (__TBB_GCC_BUILTIN_ATOMICS_PRESENT || (__TBB_GCC_VERSION >= 40102)) && !__TBB_GCC_64BIT_ATOMIC_BUILTINS_BROKEN
|
||||
return __sync_val_compare_and_swap( reinterpret_cast<volatile int64_t*>(ptr), comparand, value );
|
||||
#else /* !__TBB_GCC_BUILTIN_ATOMICS_PRESENT */
|
||||
//TODO: look like ICC 13.0 has some issues with this code, investigate it more deeply
|
||||
int64_t result;
|
||||
union {
|
||||
int64_t i64;
|
||||
int32_t i32[2];
|
||||
};
|
||||
i64 = value;
|
||||
#if __PIC__
|
||||
/* compiling position-independent code */
|
||||
// EBX register preserved for compliance with position-independent code rules on IA32
|
||||
int32_t tmp;
|
||||
__asm__ __volatile__ (
|
||||
"movl %%ebx,%2\n\t"
|
||||
"movl %5,%%ebx\n\t"
|
||||
#if __GNUC__==3
|
||||
"lock\n\t cmpxchg8b %1\n\t"
|
||||
#else
|
||||
"lock\n\t cmpxchg8b (%3)\n\t"
|
||||
#endif
|
||||
"movl %2,%%ebx"
|
||||
: "=A"(result)
|
||||
, "=m"(*(__TBB_VOLATILE int64_t *)ptr)
|
||||
, "=m"(tmp)
|
||||
#if __GNUC__==3
|
||||
: "m"(*(__TBB_VOLATILE int64_t *)ptr)
|
||||
#else
|
||||
: "SD"(ptr)
|
||||
#endif
|
||||
, "0"(comparand)
|
||||
, "m"(i32[0]), "c"(i32[1])
|
||||
: "memory"
|
||||
#if __INTEL_COMPILER
|
||||
,"ebx"
|
||||
#endif
|
||||
);
|
||||
#else /* !__PIC__ */
|
||||
__asm__ __volatile__ (
|
||||
"lock\n\t cmpxchg8b %1\n\t"
|
||||
: "=A"(result), "=m"(*(__TBB_VOLATILE int64_t *)ptr)
|
||||
: "m"(*(__TBB_VOLATILE int64_t *)ptr)
|
||||
, "0"(comparand)
|
||||
, "b"(i32[0]), "c"(i32[1])
|
||||
: "memory"
|
||||
);
|
||||
#endif /* __PIC__ */
|
||||
return result;
|
||||
#endif /* !__TBB_GCC_BUILTIN_ATOMICS_PRESENT */
|
||||
}
|
||||
|
||||
#undef __TBB_IA32_CAS8_NOINLINE
|
||||
|
||||
#if __INTEL_COMPILER
|
||||
#pragma warning( pop )
|
||||
#endif // warning 998 is back
|
||||
|
||||
static inline void __TBB_machine_or( volatile void *ptr, uint32_t addend ) {
|
||||
__asm__ __volatile__("lock\norl %1,%0" : "=m"(*(__TBB_VOLATILE uint32_t *)ptr) : "r"(addend), "m"(*(__TBB_VOLATILE uint32_t *)ptr) : "memory");
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_and( volatile void *ptr, uint32_t addend ) {
|
||||
__asm__ __volatile__("lock\nandl %1,%0" : "=m"(*(__TBB_VOLATILE uint32_t *)ptr) : "r"(addend), "m"(*(__TBB_VOLATILE uint32_t *)ptr) : "memory");
|
||||
}
|
||||
|
||||
//TODO: Check if it possible and profitable for IA-32 architecture on (Linux* and Windows*)
|
||||
//to use of 64-bit load/store via floating point registers together with full fence
|
||||
//for sequentially consistent load/store, instead of CAS.
|
||||
|
||||
#if __clang__
|
||||
#define __TBB_fildq "fildll"
|
||||
#define __TBB_fistpq "fistpll"
|
||||
#else
|
||||
#define __TBB_fildq "fildq"
|
||||
#define __TBB_fistpq "fistpq"
|
||||
#endif
|
||||
|
||||
static inline int64_t __TBB_machine_aligned_load8 (const volatile void *ptr) {
|
||||
__TBB_ASSERT(tbb::internal::is_aligned(ptr,8),"__TBB_machine_aligned_load8 should be used with 8 byte aligned locations only \n");
|
||||
int64_t result;
|
||||
__asm__ __volatile__ ( __TBB_fildq " %1\n\t"
|
||||
__TBB_fistpq " %0" : "=m"(result) : "m"(*(const __TBB_VOLATILE uint64_t*)ptr) : "memory" );
|
||||
return result;
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_aligned_store8 (volatile void *ptr, int64_t value ) {
|
||||
__TBB_ASSERT(tbb::internal::is_aligned(ptr,8),"__TBB_machine_aligned_store8 should be used with 8 byte aligned locations only \n");
|
||||
// Aligned store
|
||||
__asm__ __volatile__ ( __TBB_fildq " %1\n\t"
|
||||
__TBB_fistpq " %0" : "=m"(*(__TBB_VOLATILE int64_t*)ptr) : "m"(value) : "memory" );
|
||||
}
|
||||
|
||||
static inline int64_t __TBB_machine_load8 (const volatile void *ptr) {
|
||||
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
|
||||
if( tbb::internal::is_aligned(ptr,8)) {
|
||||
#endif
|
||||
return __TBB_machine_aligned_load8(ptr);
|
||||
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
|
||||
} else {
|
||||
// Unaligned load
|
||||
return __TBB_machine_cmpswp8(const_cast<void*>(ptr),0,0);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
//! Handles misaligned 8-byte store
|
||||
/** Defined in tbb_misc.cpp */
|
||||
extern "C" void __TBB_machine_store8_slow( volatile void *ptr, int64_t value );
|
||||
extern "C" void __TBB_machine_store8_slow_perf_warning( volatile void *ptr );
|
||||
|
||||
static inline void __TBB_machine_store8(volatile void *ptr, int64_t value) {
|
||||
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
|
||||
if( tbb::internal::is_aligned(ptr,8)) {
|
||||
#endif
|
||||
__TBB_machine_aligned_store8(ptr,value);
|
||||
#if __TBB_FORCE_64BIT_ALIGNMENT_BROKEN
|
||||
} else {
|
||||
// Unaligned store
|
||||
#if TBB_USE_PERFORMANCE_WARNINGS
|
||||
__TBB_machine_store8_slow_perf_warning(ptr);
|
||||
#endif /* TBB_USE_PERFORMANCE_WARNINGS */
|
||||
__TBB_machine_store8_slow(ptr,value);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
// Machine specific atomic operations
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
|
||||
|
||||
#define __TBB_USE_GENERIC_DWORD_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_DWORD_FETCH_STORE 1
|
||||
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_ia64_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_linux_ia64_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include <ia64intrin.h>
|
||||
|
||||
#define __TBB_WORDSIZE 8
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
|
||||
#if __INTEL_COMPILER
|
||||
#define __TBB_compiler_fence()
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper()
|
||||
#define __TBB_release_consistency_helper()
|
||||
#define __TBB_full_memory_fence() __mf()
|
||||
#else
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
// Even though GCC imbues volatile loads with acquire semantics, it sometimes moves
|
||||
// loads over the acquire fence. The following helpers stop such incorrect code motion.
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("mf": : :"memory")
|
||||
#endif /* !__INTEL_COMPILER */
|
||||
|
||||
// Most of the functions will be in a .s file
|
||||
// TODO: revise dynamic_link, memory pools and etc. if the library dependency is removed.
|
||||
|
||||
extern "C" {
|
||||
int8_t __TBB_machine_fetchadd1__TBB_full_fence (volatile void *ptr, int8_t addend);
|
||||
int8_t __TBB_machine_fetchadd1acquire(volatile void *ptr, int8_t addend);
|
||||
int8_t __TBB_machine_fetchadd1release(volatile void *ptr, int8_t addend);
|
||||
|
||||
int16_t __TBB_machine_fetchadd2__TBB_full_fence (volatile void *ptr, int16_t addend);
|
||||
int16_t __TBB_machine_fetchadd2acquire(volatile void *ptr, int16_t addend);
|
||||
int16_t __TBB_machine_fetchadd2release(volatile void *ptr, int16_t addend);
|
||||
|
||||
int32_t __TBB_machine_fetchadd4__TBB_full_fence (volatile void *ptr, int32_t value);
|
||||
int32_t __TBB_machine_fetchadd4acquire(volatile void *ptr, int32_t addend);
|
||||
int32_t __TBB_machine_fetchadd4release(volatile void *ptr, int32_t addend);
|
||||
|
||||
int64_t __TBB_machine_fetchadd8__TBB_full_fence (volatile void *ptr, int64_t value);
|
||||
int64_t __TBB_machine_fetchadd8acquire(volatile void *ptr, int64_t addend);
|
||||
int64_t __TBB_machine_fetchadd8release(volatile void *ptr, int64_t addend);
|
||||
|
||||
int8_t __TBB_machine_fetchstore1__TBB_full_fence (volatile void *ptr, int8_t value);
|
||||
int8_t __TBB_machine_fetchstore1acquire(volatile void *ptr, int8_t value);
|
||||
int8_t __TBB_machine_fetchstore1release(volatile void *ptr, int8_t value);
|
||||
|
||||
int16_t __TBB_machine_fetchstore2__TBB_full_fence (volatile void *ptr, int16_t value);
|
||||
int16_t __TBB_machine_fetchstore2acquire(volatile void *ptr, int16_t value);
|
||||
int16_t __TBB_machine_fetchstore2release(volatile void *ptr, int16_t value);
|
||||
|
||||
int32_t __TBB_machine_fetchstore4__TBB_full_fence (volatile void *ptr, int32_t value);
|
||||
int32_t __TBB_machine_fetchstore4acquire(volatile void *ptr, int32_t value);
|
||||
int32_t __TBB_machine_fetchstore4release(volatile void *ptr, int32_t value);
|
||||
|
||||
int64_t __TBB_machine_fetchstore8__TBB_full_fence (volatile void *ptr, int64_t value);
|
||||
int64_t __TBB_machine_fetchstore8acquire(volatile void *ptr, int64_t value);
|
||||
int64_t __TBB_machine_fetchstore8release(volatile void *ptr, int64_t value);
|
||||
|
||||
int8_t __TBB_machine_cmpswp1__TBB_full_fence (volatile void *ptr, int8_t value, int8_t comparand);
|
||||
int8_t __TBB_machine_cmpswp1acquire(volatile void *ptr, int8_t value, int8_t comparand);
|
||||
int8_t __TBB_machine_cmpswp1release(volatile void *ptr, int8_t value, int8_t comparand);
|
||||
|
||||
int16_t __TBB_machine_cmpswp2__TBB_full_fence (volatile void *ptr, int16_t value, int16_t comparand);
|
||||
int16_t __TBB_machine_cmpswp2acquire(volatile void *ptr, int16_t value, int16_t comparand);
|
||||
int16_t __TBB_machine_cmpswp2release(volatile void *ptr, int16_t value, int16_t comparand);
|
||||
|
||||
int32_t __TBB_machine_cmpswp4__TBB_full_fence (volatile void *ptr, int32_t value, int32_t comparand);
|
||||
int32_t __TBB_machine_cmpswp4acquire(volatile void *ptr, int32_t value, int32_t comparand);
|
||||
int32_t __TBB_machine_cmpswp4release(volatile void *ptr, int32_t value, int32_t comparand);
|
||||
|
||||
int64_t __TBB_machine_cmpswp8__TBB_full_fence (volatile void *ptr, int64_t value, int64_t comparand);
|
||||
int64_t __TBB_machine_cmpswp8acquire(volatile void *ptr, int64_t value, int64_t comparand);
|
||||
int64_t __TBB_machine_cmpswp8release(volatile void *ptr, int64_t value, int64_t comparand);
|
||||
|
||||
int64_t __TBB_machine_lg(uint64_t value);
|
||||
void __TBB_machine_pause(int32_t delay);
|
||||
bool __TBB_machine_trylockbyte( volatile unsigned char &ptr );
|
||||
int64_t __TBB_machine_lockbyte( volatile unsigned char &ptr );
|
||||
|
||||
//! Retrieves the current RSE backing store pointer. IA64 specific.
|
||||
void* __TBB_get_bsp();
|
||||
|
||||
int32_t __TBB_machine_load1_relaxed(const void *ptr);
|
||||
int32_t __TBB_machine_load2_relaxed(const void *ptr);
|
||||
int32_t __TBB_machine_load4_relaxed(const void *ptr);
|
||||
int64_t __TBB_machine_load8_relaxed(const void *ptr);
|
||||
|
||||
void __TBB_machine_store1_relaxed(void *ptr, int32_t value);
|
||||
void __TBB_machine_store2_relaxed(void *ptr, int32_t value);
|
||||
void __TBB_machine_store4_relaxed(void *ptr, int32_t value);
|
||||
void __TBB_machine_store8_relaxed(void *ptr, int64_t value);
|
||||
} // extern "C"
|
||||
|
||||
// Mapping old entry points to the names corresponding to the new full_fence identifier.
|
||||
#define __TBB_machine_fetchadd1full_fence __TBB_machine_fetchadd1__TBB_full_fence
|
||||
#define __TBB_machine_fetchadd2full_fence __TBB_machine_fetchadd2__TBB_full_fence
|
||||
#define __TBB_machine_fetchadd4full_fence __TBB_machine_fetchadd4__TBB_full_fence
|
||||
#define __TBB_machine_fetchadd8full_fence __TBB_machine_fetchadd8__TBB_full_fence
|
||||
#define __TBB_machine_fetchstore1full_fence __TBB_machine_fetchstore1__TBB_full_fence
|
||||
#define __TBB_machine_fetchstore2full_fence __TBB_machine_fetchstore2__TBB_full_fence
|
||||
#define __TBB_machine_fetchstore4full_fence __TBB_machine_fetchstore4__TBB_full_fence
|
||||
#define __TBB_machine_fetchstore8full_fence __TBB_machine_fetchstore8__TBB_full_fence
|
||||
#define __TBB_machine_cmpswp1full_fence __TBB_machine_cmpswp1__TBB_full_fence
|
||||
#define __TBB_machine_cmpswp2full_fence __TBB_machine_cmpswp2__TBB_full_fence
|
||||
#define __TBB_machine_cmpswp4full_fence __TBB_machine_cmpswp4__TBB_full_fence
|
||||
#define __TBB_machine_cmpswp8full_fence __TBB_machine_cmpswp8__TBB_full_fence
|
||||
|
||||
// Mapping relaxed operations to the entry points implementing them.
|
||||
/** On IA64 RMW operations implicitly have acquire semantics. Thus one cannot
|
||||
actually have completely relaxed RMW operation here. **/
|
||||
#define __TBB_machine_fetchadd1relaxed __TBB_machine_fetchadd1acquire
|
||||
#define __TBB_machine_fetchadd2relaxed __TBB_machine_fetchadd2acquire
|
||||
#define __TBB_machine_fetchadd4relaxed __TBB_machine_fetchadd4acquire
|
||||
#define __TBB_machine_fetchadd8relaxed __TBB_machine_fetchadd8acquire
|
||||
#define __TBB_machine_fetchstore1relaxed __TBB_machine_fetchstore1acquire
|
||||
#define __TBB_machine_fetchstore2relaxed __TBB_machine_fetchstore2acquire
|
||||
#define __TBB_machine_fetchstore4relaxed __TBB_machine_fetchstore4acquire
|
||||
#define __TBB_machine_fetchstore8relaxed __TBB_machine_fetchstore8acquire
|
||||
#define __TBB_machine_cmpswp1relaxed __TBB_machine_cmpswp1acquire
|
||||
#define __TBB_machine_cmpswp2relaxed __TBB_machine_cmpswp2acquire
|
||||
#define __TBB_machine_cmpswp4relaxed __TBB_machine_cmpswp4acquire
|
||||
#define __TBB_machine_cmpswp8relaxed __TBB_machine_cmpswp8acquire
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,V) \
|
||||
template <typename T> \
|
||||
struct machine_load_store_relaxed<T,S> { \
|
||||
static inline T load ( const T& location ) { \
|
||||
return (T)__TBB_machine_load##S##_relaxed(&location); \
|
||||
} \
|
||||
static inline void store ( T& location, T value ) { \
|
||||
__TBB_machine_store##S##_relaxed(&location, (V)value); \
|
||||
} \
|
||||
}
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t);
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t);
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t);
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t);
|
||||
}} // namespaces internal, tbb
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_ATOMICS
|
||||
|
||||
#define __TBB_USE_FENCED_ATOMICS 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
// Definition of Lock functions
|
||||
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)
|
||||
#define __TBB_LockByte(P) __TBB_machine_lockbyte(P)
|
||||
|
||||
// Definition of other utility functions
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
@@ -0,0 +1,96 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_linux_intel64_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_linux_intel64_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include "gcc_ia32_common.h"
|
||||
|
||||
#define __TBB_WORDSIZE 8
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__("": : :"memory")
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
|
||||
#ifndef __TBB_full_memory_fence
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("mfence": : :"memory")
|
||||
#endif
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,X) \
|
||||
static inline T __TBB_machine_cmpswp##S (volatile void *ptr, T value, T comparand ) \
|
||||
{ \
|
||||
T result; \
|
||||
\
|
||||
__asm__ __volatile__("lock\ncmpxchg" X " %2,%1" \
|
||||
: "=a"(result), "=m"(*(volatile T*)ptr) \
|
||||
: "q"(value), "0"(comparand), "m"(*(volatile T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
\
|
||||
static inline T __TBB_machine_fetchadd##S(volatile void *ptr, T addend) \
|
||||
{ \
|
||||
T result; \
|
||||
__asm__ __volatile__("lock\nxadd" X " %0,%1" \
|
||||
: "=r"(result),"=m"(*(volatile T*)ptr) \
|
||||
: "0"(addend), "m"(*(volatile T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
\
|
||||
static inline T __TBB_machine_fetchstore##S(volatile void *ptr, T value) \
|
||||
{ \
|
||||
T result; \
|
||||
__asm__ __volatile__("lock\nxchg" X " %0,%1" \
|
||||
: "=r"(result),"=m"(*(volatile T*)ptr) \
|
||||
: "0"(value), "m"(*(volatile T*)ptr) \
|
||||
: "memory"); \
|
||||
return result; \
|
||||
} \
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1,int8_t,"")
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2,int16_t,"")
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4,int32_t,"")
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(8,int64_t,"q")
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_ATOMICS
|
||||
|
||||
static inline void __TBB_machine_or( volatile void *ptr, uint64_t value ) {
|
||||
__asm__ __volatile__("lock\norq %1,%0" : "=m"(*(volatile uint64_t*)ptr) : "r"(value), "m"(*(volatile uint64_t*)ptr) : "memory");
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_and( volatile void *ptr, uint64_t value ) {
|
||||
__asm__ __volatile__("lock\nandq %1,%0" : "=m"(*(volatile uint64_t*)ptr) : "r"(value), "m"(*(volatile uint64_t*)ptr) : "memory");
|
||||
}
|
||||
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
|
||||
|
||||
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
@@ -0,0 +1,313 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_gcc_power_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_gcc_power_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include <unistd.h>
|
||||
|
||||
// TODO: rename to gcc_power.h?
|
||||
// This file is for Power Architecture with compilers supporting GNU inline-assembler syntax (currently GNU g++ and IBM XL).
|
||||
// Note that XL V9.0 (sometimes?) has trouble dealing with empty input and/or clobber lists, so they should be avoided.
|
||||
|
||||
#if __powerpc64__ || __ppc64__
|
||||
// IBM XL documents __powerpc64__ (and __PPC64__).
|
||||
// Apple documents __ppc64__ (with __ppc__ only on 32-bit).
|
||||
#define __TBB_WORDSIZE 8
|
||||
#else
|
||||
#define __TBB_WORDSIZE 4
|
||||
#endif
|
||||
|
||||
// Traditionally Power Architecture is big-endian.
|
||||
// Little-endian could be just an address manipulation (compatibility with TBB not verified),
|
||||
// or normal little-endian (on more recent systems). Embedded PowerPC systems may support
|
||||
// page-specific endianness, but then one endianness must be hidden from TBB so that it still sees only one.
|
||||
#if __BIG_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_BIG_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
|
||||
#elif __LITTLE_ENDIAN__ || (defined(__BYTE_ORDER__) && __BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
#elif defined(__BYTE_ORDER__)
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
|
||||
#else
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_DETECT
|
||||
#endif
|
||||
|
||||
// On Power Architecture, (lock-free) 64-bit atomics require 64-bit hardware:
|
||||
#if __TBB_WORDSIZE==8
|
||||
// Do not change the following definition, because TBB itself will use 64-bit atomics in 64-bit builds.
|
||||
#define __TBB_64BIT_ATOMICS 1
|
||||
#elif __bgp__
|
||||
// Do not change the following definition, because this is known 32-bit hardware.
|
||||
#define __TBB_64BIT_ATOMICS 0
|
||||
#else
|
||||
// To enable 64-bit atomics in 32-bit builds, set the value below to 1 instead of 0.
|
||||
// You must make certain that the program will only use them on actual 64-bit hardware
|
||||
// (which typically means that the entire program is only executed on such hardware),
|
||||
// because their implementation involves machine instructions that are illegal elsewhere.
|
||||
// The setting can be chosen independently per compilation unit,
|
||||
// which also means that TBB itself does not need to be rebuilt.
|
||||
// Alternatively (but only for the current architecture and TBB version),
|
||||
// override the default as a predefined macro when invoking the compiler.
|
||||
#ifndef __TBB_64BIT_ATOMICS
|
||||
#define __TBB_64BIT_ATOMICS 0
|
||||
#endif
|
||||
#endif
|
||||
|
||||
inline int32_t __TBB_machine_cmpswp4 (volatile void *ptr, int32_t value, int32_t comparand )
|
||||
{
|
||||
int32_t result;
|
||||
|
||||
__asm__ __volatile__("sync\n"
|
||||
"0:\n\t"
|
||||
"lwarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
|
||||
"cmpw %[res],%[cmp]\n\t" /* compare against comparand */
|
||||
"bne- 1f\n\t" /* exit if not same */
|
||||
"stwcx. %[val],0,%[ptr]\n\t" /* store new value */
|
||||
"bne- 0b\n" /* retry if reservation lost */
|
||||
"1:\n\t" /* the exit */
|
||||
"isync"
|
||||
: [res]"=&r"(result)
|
||||
, "+m"(* (int32_t*) ptr) /* redundant with "memory" */
|
||||
: [ptr]"r"(ptr)
|
||||
, [val]"r"(value)
|
||||
, [cmp]"r"(comparand)
|
||||
: "memory" /* compiler full fence */
|
||||
, "cr0" /* clobbered by cmp and/or stwcx. */
|
||||
);
|
||||
return result;
|
||||
}
|
||||
|
||||
#if __TBB_WORDSIZE==8
|
||||
|
||||
inline int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand )
|
||||
{
|
||||
int64_t result;
|
||||
__asm__ __volatile__("sync\n"
|
||||
"0:\n\t"
|
||||
"ldarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
|
||||
"cmpd %[res],%[cmp]\n\t" /* compare against comparand */
|
||||
"bne- 1f\n\t" /* exit if not same */
|
||||
"stdcx. %[val],0,%[ptr]\n\t" /* store new value */
|
||||
"bne- 0b\n" /* retry if reservation lost */
|
||||
"1:\n\t" /* the exit */
|
||||
"isync"
|
||||
: [res]"=&r"(result)
|
||||
, "+m"(* (int64_t*) ptr) /* redundant with "memory" */
|
||||
: [ptr]"r"(ptr)
|
||||
, [val]"r"(value)
|
||||
, [cmp]"r"(comparand)
|
||||
: "memory" /* compiler full fence */
|
||||
, "cr0" /* clobbered by cmp and/or stdcx. */
|
||||
);
|
||||
return result;
|
||||
}
|
||||
|
||||
#elif __TBB_64BIT_ATOMICS /* && __TBB_WORDSIZE==4 */
|
||||
|
||||
inline int64_t __TBB_machine_cmpswp8 (volatile void *ptr, int64_t value, int64_t comparand )
|
||||
{
|
||||
int64_t result;
|
||||
int64_t value_register, comparand_register, result_register; // dummy variables to allocate registers
|
||||
__asm__ __volatile__("sync\n\t"
|
||||
"ld %[val],%[valm]\n\t"
|
||||
"ld %[cmp],%[cmpm]\n"
|
||||
"0:\n\t"
|
||||
"ldarx %[res],0,%[ptr]\n\t" /* load w/ reservation */
|
||||
"cmpd %[res],%[cmp]\n\t" /* compare against comparand */
|
||||
"bne- 1f\n\t" /* exit if not same */
|
||||
"stdcx. %[val],0,%[ptr]\n\t" /* store new value */
|
||||
"bne- 0b\n" /* retry if reservation lost */
|
||||
"1:\n\t" /* the exit */
|
||||
"std %[res],%[resm]\n\t"
|
||||
"isync"
|
||||
: [resm]"=m"(result)
|
||||
, [res] "=&r"( result_register)
|
||||
, [val] "=&r"( value_register)
|
||||
, [cmp] "=&r"(comparand_register)
|
||||
, "+m"(* (int64_t*) ptr) /* redundant with "memory" */
|
||||
: [ptr] "r"(ptr)
|
||||
, [valm]"m"(value)
|
||||
, [cmpm]"m"(comparand)
|
||||
: "memory" /* compiler full fence */
|
||||
, "cr0" /* clobbered by cmpd and/or stdcx. */
|
||||
);
|
||||
return result;
|
||||
}
|
||||
|
||||
#endif /* __TBB_WORDSIZE==4 && __TBB_64BIT_ATOMICS */
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_LOAD_STORE(S,ldx,stx,cmpx) \
|
||||
template <typename T> \
|
||||
struct machine_load_store<T,S> { \
|
||||
static inline T load_with_acquire(const volatile T& location) { \
|
||||
T result; \
|
||||
__asm__ __volatile__(ldx " %[res],0(%[ptr])\n" \
|
||||
"0:\n\t" \
|
||||
cmpx " %[res],%[res]\n\t" \
|
||||
"bne- 0b\n\t" \
|
||||
"isync" \
|
||||
: [res]"=r"(result) \
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */ \
|
||||
, "m"(location) /* redundant with "memory" */ \
|
||||
: "memory" /* compiler acquire fence */ \
|
||||
, "cr0" /* clobbered by cmpw/cmpd */); \
|
||||
return result; \
|
||||
} \
|
||||
static inline void store_with_release(volatile T &location, T value) { \
|
||||
__asm__ __volatile__("lwsync\n\t" \
|
||||
stx " %[val],0(%[ptr])" \
|
||||
: "=m"(location) /* redundant with "memory" */ \
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */ \
|
||||
, [val]"r"(value) \
|
||||
: "memory"/*compiler release fence*/ /*(cr0 not affected)*/); \
|
||||
} \
|
||||
}; \
|
||||
\
|
||||
template <typename T> \
|
||||
struct machine_load_store_relaxed<T,S> { \
|
||||
static inline T load (const __TBB_atomic T& location) { \
|
||||
T result; \
|
||||
__asm__ __volatile__(ldx " %[res],0(%[ptr])" \
|
||||
: [res]"=r"(result) \
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */ \
|
||||
, "m"(location) \
|
||||
); /*(no compiler fence)*/ /*(cr0 not affected)*/ \
|
||||
return result; \
|
||||
} \
|
||||
static inline void store (__TBB_atomic T &location, T value) { \
|
||||
__asm__ __volatile__(stx " %[val],0(%[ptr])" \
|
||||
: "=m"(location) \
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */ \
|
||||
, [val]"r"(value) \
|
||||
); /*(no compiler fence)*/ /*(cr0 not affected)*/ \
|
||||
} \
|
||||
};
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
__TBB_MACHINE_DEFINE_LOAD_STORE(1,"lbz","stb","cmpw")
|
||||
__TBB_MACHINE_DEFINE_LOAD_STORE(2,"lhz","sth","cmpw")
|
||||
__TBB_MACHINE_DEFINE_LOAD_STORE(4,"lwz","stw","cmpw")
|
||||
|
||||
#if __TBB_WORDSIZE==8
|
||||
|
||||
__TBB_MACHINE_DEFINE_LOAD_STORE(8,"ld" ,"std","cmpd")
|
||||
|
||||
#elif __TBB_64BIT_ATOMICS /* && __TBB_WORDSIZE==4 */
|
||||
|
||||
template <typename T>
|
||||
struct machine_load_store<T,8> {
|
||||
static inline T load_with_acquire(const volatile T& location) {
|
||||
T result;
|
||||
T result_register; // dummy variable to allocate a register
|
||||
__asm__ __volatile__("ld %[res],0(%[ptr])\n\t"
|
||||
"std %[res],%[resm]\n"
|
||||
"0:\n\t"
|
||||
"cmpd %[res],%[res]\n\t"
|
||||
"bne- 0b\n\t"
|
||||
"isync"
|
||||
: [resm]"=m"(result)
|
||||
, [res]"=&r"(result_register)
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */
|
||||
, "m"(location) /* redundant with "memory" */
|
||||
: "memory" /* compiler acquire fence */
|
||||
, "cr0" /* clobbered by cmpd */);
|
||||
return result;
|
||||
}
|
||||
|
||||
static inline void store_with_release(volatile T &location, T value) {
|
||||
T value_register; // dummy variable to allocate a register
|
||||
__asm__ __volatile__("lwsync\n\t"
|
||||
"ld %[val],%[valm]\n\t"
|
||||
"std %[val],0(%[ptr])"
|
||||
: "=m"(location) /* redundant with "memory" */
|
||||
, [val]"=&r"(value_register)
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */
|
||||
, [valm]"m"(value)
|
||||
: "memory"/*compiler release fence*/ /*(cr0 not affected)*/);
|
||||
}
|
||||
};
|
||||
|
||||
struct machine_load_store_relaxed<T,8> {
|
||||
static inline T load (const volatile T& location) {
|
||||
T result;
|
||||
T result_register; // dummy variable to allocate a register
|
||||
__asm__ __volatile__("ld %[res],0(%[ptr])\n\t"
|
||||
"std %[res],%[resm]"
|
||||
: [resm]"=m"(result)
|
||||
, [res]"=&r"(result_register)
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */
|
||||
, "m"(location)
|
||||
); /*(no compiler fence)*/ /*(cr0 not affected)*/
|
||||
return result;
|
||||
}
|
||||
|
||||
static inline void store (volatile T &location, T value) {
|
||||
T value_register; // dummy variable to allocate a register
|
||||
__asm__ __volatile__("ld %[val],%[valm]\n\t"
|
||||
"std %[val],0(%[ptr])"
|
||||
: "=m"(location)
|
||||
, [val]"=&r"(value_register)
|
||||
: [ptr]"b"(&location) /* cannot use register 0 here */
|
||||
, [valm]"m"(value)
|
||||
); /*(no compiler fence)*/ /*(cr0 not affected)*/
|
||||
}
|
||||
};
|
||||
#define __TBB_machine_load_store_relaxed_8
|
||||
|
||||
#endif /* __TBB_WORDSIZE==4 && __TBB_64BIT_ATOMICS */
|
||||
|
||||
}} // namespaces internal, tbb
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_LOAD_STORE
|
||||
|
||||
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
|
||||
#define __TBB_USE_GENERIC_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#define __TBB_control_consistency_helper() __asm__ __volatile__("isync": : :"memory")
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__( "sync": : :"memory")
|
||||
|
||||
static inline intptr_t __TBB_machine_lg( uintptr_t x ) {
|
||||
__TBB_ASSERT(x, "__TBB_Log2(0) undefined");
|
||||
// cntlzd/cntlzw starts counting at 2^63/2^31 (ignoring any higher-order bits), and does not affect cr0
|
||||
#if __TBB_WORDSIZE==8
|
||||
__asm__ __volatile__ ("cntlzd %0,%0" : "+r"(x));
|
||||
return 63-static_cast<intptr_t>(x);
|
||||
#else
|
||||
__asm__ __volatile__ ("cntlzw %0,%0" : "+r"(x));
|
||||
return 31-static_cast<intptr_t>(x);
|
||||
#endif
|
||||
}
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
|
||||
// Assumes implicit alignment for any 32-bit value
|
||||
typedef uint32_t __TBB_Flag;
|
||||
#define __TBB_Flag __TBB_Flag
|
||||
|
||||
inline bool __TBB_machine_trylockbyte( __TBB_atomic __TBB_Flag &flag ) {
|
||||
return __TBB_machine_cmpswp4(&flag,1,0)==0;
|
||||
}
|
||||
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_macos_common_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_macos_common_H
|
||||
|
||||
#include <sched.h>
|
||||
#define __TBB_Yield() sched_yield()
|
||||
|
||||
// __TBB_HardwareConcurrency
|
||||
|
||||
#include <sys/types.h>
|
||||
#include <sys/sysctl.h>
|
||||
|
||||
static inline int __TBB_macos_available_cpu() {
|
||||
int name[2] = {CTL_HW, HW_AVAILCPU};
|
||||
int ncpu;
|
||||
size_t size = sizeof(ncpu);
|
||||
sysctl( name, 2, &ncpu, &size, NULL, 0 );
|
||||
return ncpu;
|
||||
}
|
||||
|
||||
#define __TBB_HardwareConcurrency() __TBB_macos_available_cpu()
|
||||
|
||||
#ifndef __TBB_full_memory_fence
|
||||
// TBB has not recognized the architecture (none of the architecture abstraction
|
||||
// headers was included).
|
||||
#define __TBB_UnknownArchitecture 1
|
||||
#endif
|
||||
|
||||
#if __TBB_UnknownArchitecture
|
||||
// Implementation of atomic operations based on OS provided primitives
|
||||
#include <libkern/OSAtomic.h>
|
||||
|
||||
static inline int64_t __TBB_machine_cmpswp8_OsX(volatile void *ptr, int64_t value, int64_t comparand)
|
||||
{
|
||||
__TBB_ASSERT( tbb::internal::is_aligned(ptr,8), "address not properly aligned for OS X* atomics");
|
||||
int64_t* address = (int64_t*)ptr;
|
||||
while( !OSAtomicCompareAndSwap64Barrier(comparand, value, address) ){
|
||||
#if __TBB_WORDSIZE==8
|
||||
int64_t snapshot = *address;
|
||||
#else
|
||||
int64_t snapshot = OSAtomicAdd64( 0, address );
|
||||
#endif
|
||||
if( snapshot!=comparand ) return snapshot;
|
||||
}
|
||||
return comparand;
|
||||
}
|
||||
|
||||
#define __TBB_machine_cmpswp8 __TBB_machine_cmpswp8_OsX
|
||||
|
||||
#endif /* __TBB_UnknownArchitecture */
|
||||
|
||||
#if __TBB_UnknownArchitecture
|
||||
|
||||
#ifndef __TBB_WORDSIZE
|
||||
#define __TBB_WORDSIZE 4
|
||||
#endif
|
||||
|
||||
#ifdef __TBB_ENDIANNESS
|
||||
// Already determined based on hardware architecture.
|
||||
#elif __BIG_ENDIAN__
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
|
||||
#elif __LITTLE_ENDIAN__
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
#else
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
|
||||
#endif
|
||||
|
||||
/** As this generic implementation has absolutely no information about underlying
|
||||
hardware, its performance most likely will be sub-optimal because of full memory
|
||||
fence usages where a more lightweight synchronization means (or none at all)
|
||||
could suffice. Thus if you use this header to enable TBB on a new platform,
|
||||
consider forking it and relaxing below helpers as appropriate. **/
|
||||
#define __TBB_control_consistency_helper() OSMemoryBarrier()
|
||||
#define __TBB_acquire_consistency_helper() OSMemoryBarrier()
|
||||
#define __TBB_release_consistency_helper() OSMemoryBarrier()
|
||||
#define __TBB_full_memory_fence() OSMemoryBarrier()
|
||||
|
||||
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand)
|
||||
{
|
||||
__TBB_ASSERT( tbb::internal::is_aligned(ptr,4), "address not properly aligned for OS X* atomics");
|
||||
int32_t* address = (int32_t*)ptr;
|
||||
while( !OSAtomicCompareAndSwap32Barrier(comparand, value, address) ){
|
||||
int32_t snapshot = *address;
|
||||
if( snapshot!=comparand ) return snapshot;
|
||||
}
|
||||
return comparand;
|
||||
}
|
||||
|
||||
static inline int32_t __TBB_machine_fetchadd4(volatile void *ptr, int32_t addend)
|
||||
{
|
||||
__TBB_ASSERT( tbb::internal::is_aligned(ptr,4), "address not properly aligned for OS X* atomics");
|
||||
return OSAtomicAdd32Barrier(addend, (int32_t*)ptr) - addend;
|
||||
}
|
||||
|
||||
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend)
|
||||
{
|
||||
__TBB_ASSERT( tbb::internal::is_aligned(ptr,8), "address not properly aligned for OS X* atomics");
|
||||
return OSAtomicAdd64Barrier(addend, (int64_t*)ptr) - addend;
|
||||
}
|
||||
|
||||
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#if __TBB_WORDSIZE == 4
|
||||
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
|
||||
#endif
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#endif /* __TBB_UnknownArchitecture */
|
||||
@@ -0,0 +1,61 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_mic_common_H
|
||||
#define __TBB_mic_common_H
|
||||
|
||||
#ifndef __TBB_machine_H
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#if ! __TBB_DEFINE_MIC
|
||||
#error mic_common.h should be included only when building for Intel(R) Many Integrated Core Architecture
|
||||
#endif
|
||||
|
||||
#ifndef __TBB_PREFETCHING
|
||||
#define __TBB_PREFETCHING 1
|
||||
#endif
|
||||
#if __TBB_PREFETCHING
|
||||
#include <immintrin.h>
|
||||
#define __TBB_cl_prefetch(p) _mm_prefetch((const char*)p, _MM_HINT_T1)
|
||||
#define __TBB_cl_evict(p) _mm_clevict(p, _MM_HINT_T1)
|
||||
#endif
|
||||
|
||||
/** Intel(R) Many Integrated Core Architecture does not support mfence and pause instructions **/
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("lock; addl $0,(%%rsp)":::"memory")
|
||||
#define __TBB_Pause(x) _mm_delay_32(16*(x))
|
||||
#define __TBB_STEALING_PAUSE 1500/16
|
||||
#include <sched.h>
|
||||
#define __TBB_Yield() sched_yield()
|
||||
|
||||
// low-level timing intrinsic and its type
|
||||
#define __TBB_machine_time_stamp() _rdtsc()
|
||||
typedef uint64_t machine_tsc_t;
|
||||
|
||||
/** Specifics **/
|
||||
#define __TBB_STEALING_ABORT_ON_CONTENTION 1
|
||||
#define __TBB_YIELD2P 1
|
||||
#define __TBB_HOARD_NONLOCAL_TASKS 1
|
||||
|
||||
#if ! ( __FreeBSD__ || __linux__ )
|
||||
#error Intel(R) Many Integrated Core Compiler does not define __FreeBSD__ or __linux__ anymore. Check for the __TBB_XXX_BROKEN defined under __FreeBSD__ or __linux__.
|
||||
#endif /* ! ( __FreeBSD__ || __linux__ ) */
|
||||
|
||||
#endif /* __TBB_mic_common_H */
|
||||
@@ -0,0 +1,171 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_msvc_armv7_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_msvc_armv7_H
|
||||
|
||||
#include <intrin.h>
|
||||
#include <float.h>
|
||||
|
||||
#define __TBB_WORDSIZE 4
|
||||
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_UNSUPPORTED
|
||||
|
||||
#if defined(TBB_WIN32_USE_CL_BUILTINS)
|
||||
// We can test this on _M_IX86
|
||||
#pragma intrinsic(_ReadWriteBarrier)
|
||||
#pragma intrinsic(_mm_mfence)
|
||||
#define __TBB_compiler_fence() _ReadWriteBarrier()
|
||||
#define __TBB_full_memory_fence() _mm_mfence()
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
#else
|
||||
//Now __dmb(_ARM_BARRIER_SY) is used for both compiler and memory fences
|
||||
//This might be changed later after testing
|
||||
#define __TBB_compiler_fence() __dmb(_ARM_BARRIER_SY)
|
||||
#define __TBB_full_memory_fence() __dmb(_ARM_BARRIER_SY)
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_full_memory_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_full_memory_fence()
|
||||
#endif
|
||||
|
||||
//--------------------------------------------------
|
||||
// Compare and swap
|
||||
//--------------------------------------------------
|
||||
|
||||
/**
|
||||
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
|
||||
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
|
||||
* @param value value to assign *ptr to if *ptr==comparand
|
||||
* @param comparand value to compare with *ptr
|
||||
* @return value originally in memory at ptr, regardless of success
|
||||
*/
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(S,T,F) \
|
||||
inline T __TBB_machine_cmpswp##S( volatile void *ptr, T value, T comparand ) { \
|
||||
return _InterlockedCompareExchange##F(reinterpret_cast<volatile T *>(ptr),value,comparand); \
|
||||
} \
|
||||
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(S,T,F) \
|
||||
inline T __TBB_machine_fetchadd##S( volatile void *ptr, T value ) { \
|
||||
return _InterlockedExchangeAdd##F(reinterpret_cast<volatile T *>(ptr),value); \
|
||||
} \
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(1,char,8)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(2,short,16)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(4,long,)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_CMPSWP(8,__int64,64)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(4,long,)
|
||||
#if defined(TBB_WIN32_USE_CL_BUILTINS)
|
||||
// No _InterlockedExchangeAdd64 intrinsic on _M_IX86
|
||||
#define __TBB_64BIT_ATOMICS 0
|
||||
#else
|
||||
__TBB_MACHINE_DEFINE_ATOMICS_FETCHADD(8,__int64,64)
|
||||
#endif
|
||||
|
||||
inline void __TBB_machine_pause (int32_t delay )
|
||||
{
|
||||
while(delay>0)
|
||||
{
|
||||
__TBB_compiler_fence();
|
||||
delay--;
|
||||
}
|
||||
}
|
||||
|
||||
// API to retrieve/update FPU control setting
|
||||
#define __TBB_CPU_CTL_ENV_PRESENT 1
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
|
||||
template <typename T, size_t S>
|
||||
struct machine_load_store_relaxed {
|
||||
static inline T load ( const volatile T& location ) {
|
||||
const T value = location;
|
||||
|
||||
/*
|
||||
* An extra memory barrier is required for errata #761319
|
||||
* Please see http://infocenter.arm.com/help/topic/com.arm.doc.uan0004a
|
||||
*/
|
||||
__TBB_acquire_consistency_helper();
|
||||
return value;
|
||||
}
|
||||
|
||||
static inline void store ( volatile T& location, T value ) {
|
||||
location = value;
|
||||
}
|
||||
};
|
||||
|
||||
class cpu_ctl_env {
|
||||
private:
|
||||
unsigned int my_ctl;
|
||||
public:
|
||||
bool operator!=( const cpu_ctl_env& ctl ) const { return my_ctl != ctl.my_ctl; }
|
||||
void get_env() { my_ctl = _control87(0, 0); }
|
||||
void set_env() const { _control87( my_ctl, ~0U ); }
|
||||
};
|
||||
|
||||
} // namespace internal
|
||||
} // namespaces tbb
|
||||
|
||||
// Machine specific atomic operations
|
||||
#define __TBB_CompareAndSwap4(P,V,C) __TBB_machine_cmpswp4(P,V,C)
|
||||
#define __TBB_CompareAndSwap8(P,V,C) __TBB_machine_cmpswp8(P,V,C)
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
|
||||
// Use generics for some things
|
||||
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_DWORD_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#if defined(TBB_WIN32_USE_CL_BUILTINS)
|
||||
#if !__TBB_WIN8UI_SUPPORT
|
||||
extern "C" __declspec(dllimport) int __stdcall SwitchToThread( void );
|
||||
#define __TBB_Yield() SwitchToThread()
|
||||
#else
|
||||
#include<thread>
|
||||
#define __TBB_Yield() std::this_thread::yield()
|
||||
#endif
|
||||
#else
|
||||
#define __TBB_Yield() __yield()
|
||||
#endif
|
||||
|
||||
// Machine specific atomic operations
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_OR(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_AND(P,V)
|
||||
|
||||
template <typename T1,typename T2>
|
||||
inline void __TBB_machine_OR( T1 *operand, T2 addend ) {
|
||||
_InterlockedOr((long volatile *)operand, (long)addend);
|
||||
}
|
||||
|
||||
template <typename T1,typename T2>
|
||||
inline void __TBB_machine_AND( T1 *operand, T2 addend ) {
|
||||
_InterlockedAnd((long volatile *)operand, (long)addend);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_machine_msvc_ia32_common_H
|
||||
#define __TBB_machine_msvc_ia32_common_H
|
||||
|
||||
#include <intrin.h>
|
||||
|
||||
//TODO: consider moving this macro to tbb_config.h and used there MSVC asm is used
|
||||
#if !_M_X64 || __INTEL_COMPILER
|
||||
#define __TBB_X86_MSVC_INLINE_ASM_AVAILABLE 1
|
||||
|
||||
#if _M_X64
|
||||
#define __TBB_r(reg_name) r##reg_name
|
||||
#else
|
||||
#define __TBB_r(reg_name) e##reg_name
|
||||
#endif
|
||||
#else
|
||||
//MSVC in x64 mode does not accept inline assembler
|
||||
#define __TBB_X86_MSVC_INLINE_ASM_AVAILABLE 0
|
||||
#endif
|
||||
|
||||
#define __TBB_NO_X86_MSVC_INLINE_ASM_MSG "The compiler being used is not supported (outdated?)"
|
||||
|
||||
#if (_MSC_VER >= 1300) || (__INTEL_COMPILER) //Use compiler intrinsic when available
|
||||
#define __TBB_PAUSE_USE_INTRINSIC 1
|
||||
#pragma intrinsic(_mm_pause)
|
||||
namespace tbb { namespace internal { namespace intrinsics { namespace msvc {
|
||||
static inline void __TBB_machine_pause (uintptr_t delay ) {
|
||||
for (;delay>0; --delay )
|
||||
_mm_pause();
|
||||
}
|
||||
}}}}
|
||||
#else
|
||||
#if !__TBB_X86_MSVC_INLINE_ASM_AVAILABLE
|
||||
#error __TBB_NO_X86_MSVC_INLINE_ASM_MSG
|
||||
#endif
|
||||
|
||||
namespace tbb { namespace internal { namespace inline_asm { namespace msvc {
|
||||
static inline void __TBB_machine_pause (uintptr_t delay ) {
|
||||
_asm
|
||||
{
|
||||
mov __TBB_r(ax), delay
|
||||
__TBB_L1:
|
||||
pause
|
||||
add __TBB_r(ax), -1
|
||||
jne __TBB_L1
|
||||
}
|
||||
return;
|
||||
}
|
||||
}}}}
|
||||
#endif
|
||||
|
||||
static inline void __TBB_machine_pause (uintptr_t delay ){
|
||||
#if __TBB_PAUSE_USE_INTRINSIC
|
||||
tbb::internal::intrinsics::msvc::__TBB_machine_pause(delay);
|
||||
#else
|
||||
tbb::internal::inline_asm::msvc::__TBB_machine_pause(delay);
|
||||
#endif
|
||||
}
|
||||
|
||||
//TODO: move this function to windows_api.h or to place where it is used
|
||||
#if (_MSC_VER<1400) && (!_WIN64) && (__TBB_X86_MSVC_INLINE_ASM_AVAILABLE)
|
||||
static inline void* __TBB_machine_get_current_teb () {
|
||||
void* pteb;
|
||||
__asm mov eax, fs:[0x18]
|
||||
__asm mov pteb, eax
|
||||
return pteb;
|
||||
}
|
||||
#endif
|
||||
|
||||
#if ( _MSC_VER>=1400 && !defined(__INTEL_COMPILER) ) || (__INTEL_COMPILER>=1200)
|
||||
// MSVC did not have this intrinsic prior to VC8.
|
||||
// ICL 11.1 fails to compile a TBB example if __TBB_Log2 uses the intrinsic.
|
||||
#define __TBB_LOG2_USE_BSR_INTRINSIC 1
|
||||
#if _M_X64
|
||||
#define __TBB_BSR_INTRINSIC _BitScanReverse64
|
||||
#else
|
||||
#define __TBB_BSR_INTRINSIC _BitScanReverse
|
||||
#endif
|
||||
#pragma intrinsic(__TBB_BSR_INTRINSIC)
|
||||
|
||||
namespace tbb { namespace internal { namespace intrinsics { namespace msvc {
|
||||
inline uintptr_t __TBB_machine_lg( uintptr_t i ){
|
||||
unsigned long j;
|
||||
__TBB_BSR_INTRINSIC( &j, i );
|
||||
return j;
|
||||
}
|
||||
}}}}
|
||||
#else
|
||||
#if !__TBB_X86_MSVC_INLINE_ASM_AVAILABLE
|
||||
#error __TBB_NO_X86_MSVC_INLINE_ASM_MSG
|
||||
#endif
|
||||
|
||||
namespace tbb { namespace internal { namespace inline_asm { namespace msvc {
|
||||
inline uintptr_t __TBB_machine_lg( uintptr_t i ){
|
||||
uintptr_t j;
|
||||
__asm
|
||||
{
|
||||
bsr __TBB_r(ax), i
|
||||
mov j, __TBB_r(ax)
|
||||
}
|
||||
return j;
|
||||
}
|
||||
}}}}
|
||||
#endif
|
||||
|
||||
static inline intptr_t __TBB_machine_lg( uintptr_t i ) {
|
||||
#if __TBB_LOG2_USE_BSR_INTRINSIC
|
||||
return tbb::internal::intrinsics::msvc::__TBB_machine_lg(i);
|
||||
#else
|
||||
return tbb::internal::inline_asm::msvc::__TBB_machine_lg(i);
|
||||
#endif
|
||||
}
|
||||
|
||||
// API to retrieve/update FPU control setting
|
||||
#define __TBB_CPU_CTL_ENV_PRESENT 1
|
||||
|
||||
namespace tbb { namespace internal { class cpu_ctl_env; } }
|
||||
#if __TBB_X86_MSVC_INLINE_ASM_AVAILABLE
|
||||
inline void __TBB_get_cpu_ctl_env ( tbb::internal::cpu_ctl_env* ctl ) {
|
||||
__asm {
|
||||
__asm mov __TBB_r(ax), ctl
|
||||
__asm stmxcsr [__TBB_r(ax)]
|
||||
__asm fstcw [__TBB_r(ax)+4]
|
||||
}
|
||||
}
|
||||
inline void __TBB_set_cpu_ctl_env ( const tbb::internal::cpu_ctl_env* ctl ) {
|
||||
__asm {
|
||||
__asm mov __TBB_r(ax), ctl
|
||||
__asm ldmxcsr [__TBB_r(ax)]
|
||||
__asm fldcw [__TBB_r(ax)+4]
|
||||
}
|
||||
}
|
||||
#else
|
||||
extern "C" {
|
||||
void __TBB_EXPORTED_FUNC __TBB_get_cpu_ctl_env ( tbb::internal::cpu_ctl_env* );
|
||||
void __TBB_EXPORTED_FUNC __TBB_set_cpu_ctl_env ( const tbb::internal::cpu_ctl_env* );
|
||||
}
|
||||
#endif
|
||||
|
||||
namespace tbb {
|
||||
namespace internal {
|
||||
class cpu_ctl_env {
|
||||
private:
|
||||
int mxcsr;
|
||||
short x87cw;
|
||||
static const int MXCSR_CONTROL_MASK = ~0x3f; /* all except last six status bits */
|
||||
public:
|
||||
bool operator!=( const cpu_ctl_env& ctl ) const { return mxcsr != ctl.mxcsr || x87cw != ctl.x87cw; }
|
||||
void get_env() {
|
||||
__TBB_get_cpu_ctl_env( this );
|
||||
mxcsr &= MXCSR_CONTROL_MASK;
|
||||
}
|
||||
void set_env() const { __TBB_set_cpu_ctl_env( this ); }
|
||||
};
|
||||
} // namespace internal
|
||||
} // namespace tbb
|
||||
|
||||
#if !__TBB_WIN8UI_SUPPORT
|
||||
extern "C" __declspec(dllimport) int __stdcall SwitchToThread( void );
|
||||
#define __TBB_Yield() SwitchToThread()
|
||||
#else
|
||||
#include<thread>
|
||||
#define __TBB_Yield() std::this_thread::yield()
|
||||
#endif
|
||||
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
|
||||
#undef __TBB_r
|
||||
|
||||
extern "C" {
|
||||
__int8 __TBB_EXPORTED_FUNC __TBB_machine_try_lock_elided (volatile void* ptr);
|
||||
void __TBB_EXPORTED_FUNC __TBB_machine_unlock_elided (volatile void* ptr);
|
||||
|
||||
// 'pause' instruction aborts HLE/RTM transactions
|
||||
#if __TBB_PAUSE_USE_INTRINSIC
|
||||
inline static void __TBB_machine_try_lock_elided_cancel() { _mm_pause(); }
|
||||
#else
|
||||
inline static void __TBB_machine_try_lock_elided_cancel() { _asm pause; }
|
||||
#endif
|
||||
|
||||
#if __TBB_TSX_INTRINSICS_PRESENT
|
||||
#define __TBB_machine_is_in_transaction _xtest
|
||||
#define __TBB_machine_begin_transaction _xbegin
|
||||
#define __TBB_machine_end_transaction _xend
|
||||
// The value (0xFF) below comes from the
|
||||
// Intel(R) 64 and IA-32 Architectures Optimization Reference Manual 12.4.5 lock not free
|
||||
#define __TBB_machine_transaction_conflict_abort() _xabort(0xFF)
|
||||
#else
|
||||
__int8 __TBB_EXPORTED_FUNC __TBB_machine_is_in_transaction();
|
||||
unsigned __int32 __TBB_EXPORTED_FUNC __TBB_machine_begin_transaction();
|
||||
void __TBB_EXPORTED_FUNC __TBB_machine_end_transaction();
|
||||
void __TBB_EXPORTED_FUNC __TBB_machine_transaction_conflict_abort();
|
||||
#endif /* __TBB_TSX_INTRINSICS_PRESENT */
|
||||
}
|
||||
|
||||
#endif /* __TBB_machine_msvc_ia32_common_H */
|
||||
@@ -0,0 +1,203 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_sunos_sparc_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_sunos_sparc_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#define __TBB_WORDSIZE 8
|
||||
// Big endian is assumed for SPARC.
|
||||
// While hardware may support page-specific bi-endianness, only big endian pages may be exposed to TBB
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_BIG
|
||||
|
||||
/** To those working on SPARC hardware. Consider relaxing acquire and release
|
||||
consistency helpers to no-op (as this port covers TSO mode only). **/
|
||||
#define __TBB_compiler_fence() __asm__ __volatile__ ("": : :"memory")
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_full_memory_fence() __asm__ __volatile__("membar #LoadLoad|#LoadStore|#StoreStore|#StoreLoad": : : "memory")
|
||||
|
||||
//--------------------------------------------------
|
||||
// Compare and swap
|
||||
//--------------------------------------------------
|
||||
|
||||
/**
|
||||
* Atomic CAS for 32 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
|
||||
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
|
||||
* @param value value to assign *ptr to if *ptr==comparand
|
||||
* @param comparand value to compare with *ptr
|
||||
( @return value originally in memory at ptr, regardless of success
|
||||
*/
|
||||
static inline int32_t __TBB_machine_cmpswp4(volatile void *ptr, int32_t value, int32_t comparand ){
|
||||
int32_t result;
|
||||
__asm__ __volatile__(
|
||||
"cas\t[%5],%4,%1"
|
||||
: "=m"(*(int32_t *)ptr), "=r"(result)
|
||||
: "m"(*(int32_t *)ptr), "1"(value), "r"(comparand), "r"(ptr)
|
||||
: "memory");
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomic CAS for 64 bit values, if *ptr==comparand, then *ptr=value, returns *ptr
|
||||
* @param ptr pointer to value in memory to be swapped with value if *ptr==comparand
|
||||
* @param value value to assign *ptr to if *ptr==comparand
|
||||
* @param comparand value to compare with *ptr
|
||||
( @return value originally in memory at ptr, regardless of success
|
||||
*/
|
||||
static inline int64_t __TBB_machine_cmpswp8(volatile void *ptr, int64_t value, int64_t comparand ){
|
||||
int64_t result;
|
||||
__asm__ __volatile__(
|
||||
"casx\t[%5],%4,%1"
|
||||
: "=m"(*(int64_t *)ptr), "=r"(result)
|
||||
: "m"(*(int64_t *)ptr), "1"(value), "r"(comparand), "r"(ptr)
|
||||
: "memory");
|
||||
return result;
|
||||
}
|
||||
|
||||
//---------------------------------------------------
|
||||
// Fetch and add
|
||||
//---------------------------------------------------
|
||||
|
||||
/**
|
||||
* Atomic fetch and add for 32 bit values, in this case implemented by continuously checking success of atomicity
|
||||
* @param ptr pointer to value to add addend to
|
||||
* @param addened value to add to *ptr
|
||||
* @return value at ptr before addened was added
|
||||
*/
|
||||
static inline int32_t __TBB_machine_fetchadd4(volatile void *ptr, int32_t addend){
|
||||
int32_t result;
|
||||
__asm__ __volatile__ (
|
||||
"0:\t add\t %3, %4, %0\n" // do addition
|
||||
"\t cas\t [%2], %3, %0\n" // cas to store result in memory
|
||||
"\t cmp\t %3, %0\n" // check if value from memory is original
|
||||
"\t bne,a,pn\t %%icc, 0b\n" // if not try again
|
||||
"\t mov %0, %3\n" // use branch delay slot to move new value in memory to be added
|
||||
: "=&r"(result), "=m"(*(int32_t *)ptr)
|
||||
: "r"(ptr), "r"(*(int32_t *)ptr), "r"(addend), "m"(*(int32_t *)ptr)
|
||||
: "ccr", "memory");
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomic fetch and add for 64 bit values, in this case implemented by continuously checking success of atomicity
|
||||
* @param ptr pointer to value to add addend to
|
||||
* @param addened value to add to *ptr
|
||||
* @return value at ptr before addened was added
|
||||
*/
|
||||
static inline int64_t __TBB_machine_fetchadd8(volatile void *ptr, int64_t addend){
|
||||
int64_t result;
|
||||
__asm__ __volatile__ (
|
||||
"0:\t add\t %3, %4, %0\n" // do addition
|
||||
"\t casx\t [%2], %3, %0\n" // cas to store result in memory
|
||||
"\t cmp\t %3, %0\n" // check if value from memory is original
|
||||
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
|
||||
"\t mov %0, %3\n" // use branch delay slot to move new value in memory to be added
|
||||
: "=&r"(result), "=m"(*(int64_t *)ptr)
|
||||
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(addend), "m"(*(int64_t *)ptr)
|
||||
: "ccr", "memory");
|
||||
return result;
|
||||
}
|
||||
|
||||
//--------------------------------------------------------
|
||||
// Logarithm (base two, integer)
|
||||
//--------------------------------------------------------
|
||||
|
||||
static inline int64_t __TBB_machine_lg( uint64_t x ) {
|
||||
__TBB_ASSERT(x, "__TBB_Log2(0) undefined");
|
||||
uint64_t count;
|
||||
// one hot encode
|
||||
x |= (x >> 1);
|
||||
x |= (x >> 2);
|
||||
x |= (x >> 4);
|
||||
x |= (x >> 8);
|
||||
x |= (x >> 16);
|
||||
x |= (x >> 32);
|
||||
// count 1's
|
||||
__asm__ ("popc %1, %0" : "=r"(count) : "r"(x) );
|
||||
return count-1;
|
||||
}
|
||||
|
||||
//--------------------------------------------------------
|
||||
|
||||
static inline void __TBB_machine_or( volatile void *ptr, uint64_t value ) {
|
||||
__asm__ __volatile__ (
|
||||
"0:\t or\t %2, %3, %%g1\n" // do operation
|
||||
"\t casx\t [%1], %2, %%g1\n" // cas to store result in memory
|
||||
"\t cmp\t %2, %%g1\n" // check if value from memory is original
|
||||
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
|
||||
"\t mov %%g1, %2\n" // use branch delay slot to move new value in memory to be added
|
||||
: "=m"(*(int64_t *)ptr)
|
||||
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(value), "m"(*(int64_t *)ptr)
|
||||
: "ccr", "g1", "memory");
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_and( volatile void *ptr, uint64_t value ) {
|
||||
__asm__ __volatile__ (
|
||||
"0:\t and\t %2, %3, %%g1\n" // do operation
|
||||
"\t casx\t [%1], %2, %%g1\n" // cas to store result in memory
|
||||
"\t cmp\t %2, %%g1\n" // check if value from memory is original
|
||||
"\t bne,a,pn\t %%xcc, 0b\n" // if not try again
|
||||
"\t mov %%g1, %2\n" // use branch delay slot to move new value in memory to be added
|
||||
: "=m"(*(int64_t *)ptr)
|
||||
: "r"(ptr), "r"(*(int64_t *)ptr), "r"(value), "m"(*(int64_t *)ptr)
|
||||
: "ccr", "g1", "memory");
|
||||
}
|
||||
|
||||
|
||||
static inline void __TBB_machine_pause( int32_t delay ) {
|
||||
// do nothing, inlined, doesn't matter
|
||||
}
|
||||
|
||||
// put 0xff in memory location, return memory value,
|
||||
// generic trylockbyte puts 0x01, however this is fine
|
||||
// because all that matters is that 0 is unlocked
|
||||
static inline bool __TBB_machine_trylockbyte(unsigned char &flag){
|
||||
unsigned char result;
|
||||
__asm__ __volatile__ (
|
||||
"ldstub\t [%2], %0\n"
|
||||
: "=r"(result), "=m"(flag)
|
||||
: "r"(&flag), "m"(flag)
|
||||
: "memory");
|
||||
return result == 0;
|
||||
}
|
||||
|
||||
#define __TBB_USE_GENERIC_PART_WORD_CAS 1
|
||||
#define __TBB_USE_GENERIC_PART_WORD_FETCH_ADD 1
|
||||
#define __TBB_USE_GENERIC_FETCH_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_or(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_and(P,V)
|
||||
|
||||
// Definition of other functions
|
||||
#define __TBB_Pause(V) __TBB_machine_pause(V)
|
||||
#define __TBB_Log2(V) __TBB_machine_lg(V)
|
||||
|
||||
#define __TBB_TryLockByte(P) __TBB_machine_trylockbyte(P)
|
||||
@@ -0,0 +1,79 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#ifndef __TBB_machine_windows_api_H
|
||||
#define __TBB_machine_windows_api_H
|
||||
|
||||
#if _WIN32 || _WIN64
|
||||
|
||||
#if _XBOX
|
||||
|
||||
#define NONET
|
||||
#define NOD3D
|
||||
#include <xtl.h>
|
||||
|
||||
#else // Assume "usual" Windows
|
||||
|
||||
#include <windows.h>
|
||||
|
||||
#endif // _XBOX
|
||||
|
||||
#if _WIN32_WINNT < 0x0600
|
||||
// The following Windows API function is declared explicitly;
|
||||
// otherwise it fails to compile by VS2005.
|
||||
#if !defined(WINBASEAPI) || (_WIN32_WINNT < 0x0501 && _MSC_VER == 1400)
|
||||
#define __TBB_WINBASEAPI extern "C"
|
||||
#else
|
||||
#define __TBB_WINBASEAPI WINBASEAPI
|
||||
#endif
|
||||
__TBB_WINBASEAPI BOOL WINAPI TryEnterCriticalSection( LPCRITICAL_SECTION );
|
||||
__TBB_WINBASEAPI BOOL WINAPI InitializeCriticalSectionAndSpinCount( LPCRITICAL_SECTION, DWORD );
|
||||
// Overloading WINBASEAPI macro and using local functions missing in Windows XP/2003
|
||||
#define InitializeCriticalSectionEx inlineInitializeCriticalSectionEx
|
||||
#define CreateSemaphoreEx inlineCreateSemaphoreEx
|
||||
#define CreateEventEx inlineCreateEventEx
|
||||
inline BOOL WINAPI inlineInitializeCriticalSectionEx( LPCRITICAL_SECTION lpCriticalSection, DWORD dwSpinCount, DWORD )
|
||||
{
|
||||
return InitializeCriticalSectionAndSpinCount( lpCriticalSection, dwSpinCount );
|
||||
}
|
||||
inline HANDLE WINAPI inlineCreateSemaphoreEx( LPSECURITY_ATTRIBUTES lpSemaphoreAttributes, LONG lInitialCount, LONG lMaximumCount, LPCTSTR lpName, DWORD, DWORD )
|
||||
{
|
||||
return CreateSemaphore( lpSemaphoreAttributes, lInitialCount, lMaximumCount, lpName );
|
||||
}
|
||||
inline HANDLE WINAPI inlineCreateEventEx( LPSECURITY_ATTRIBUTES lpEventAttributes, LPCTSTR lpName, DWORD dwFlags, DWORD )
|
||||
{
|
||||
BOOL manual_reset = dwFlags&0x00000001 ? TRUE : FALSE; // CREATE_EVENT_MANUAL_RESET
|
||||
BOOL initial_set = dwFlags&0x00000002 ? TRUE : FALSE; // CREATE_EVENT_INITIAL_SET
|
||||
return CreateEvent( lpEventAttributes, manual_reset, initial_set, lpName );
|
||||
}
|
||||
#endif
|
||||
|
||||
#if defined(RTL_SRWLOCK_INIT)
|
||||
#ifndef __TBB_USE_SRWLOCK
|
||||
// TODO: turn it on when bug 1952 will be fixed
|
||||
#define __TBB_USE_SRWLOCK 0
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#else
|
||||
#error tbb/machine/windows_api.h should only be used for Windows based platforms
|
||||
#endif // _WIN32 || _WIN64
|
||||
|
||||
#endif // __TBB_machine_windows_api_H
|
||||
@@ -0,0 +1,144 @@
|
||||
/*
|
||||
Copyright 2005-2014 Intel Corporation. All Rights Reserved.
|
||||
|
||||
This file is part of Threading Building Blocks. Threading Building Blocks is free software;
|
||||
you can redistribute it and/or modify it under the terms of the GNU General Public License
|
||||
version 2 as published by the Free Software Foundation. Threading Building Blocks is
|
||||
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
|
||||
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
See the GNU General Public License for more details. You should have received a copy of
|
||||
the GNU General Public License along with Threading Building Blocks; if not, write to the
|
||||
Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
||||
|
||||
As a special exception, you may use this file as part of a free software library without
|
||||
restriction. Specifically, if other files instantiate templates or use macros or inline
|
||||
functions from this file, or you compile this file and link it with other files to produce
|
||||
an executable, this file does not by itself cause the resulting executable to be covered
|
||||
by the GNU General Public License. This exception does not however invalidate any other
|
||||
reasons why the executable file might be covered by the GNU General Public License.
|
||||
*/
|
||||
|
||||
#if !defined(__TBB_machine_H) || defined(__TBB_machine_windows_ia32_H)
|
||||
#error Do not #include this internal file directly; use public TBB headers instead.
|
||||
#endif
|
||||
|
||||
#define __TBB_machine_windows_ia32_H
|
||||
|
||||
#include "msvc_ia32_common.h"
|
||||
|
||||
#define __TBB_WORDSIZE 4
|
||||
#define __TBB_ENDIANNESS __TBB_ENDIAN_LITTLE
|
||||
|
||||
#if __INTEL_COMPILER && (__INTEL_COMPILER < 1100)
|
||||
#define __TBB_compiler_fence() __asm { __asm nop }
|
||||
#define __TBB_full_memory_fence() __asm { __asm mfence }
|
||||
#elif _MSC_VER >= 1300 || __INTEL_COMPILER
|
||||
#pragma intrinsic(_ReadWriteBarrier)
|
||||
#pragma intrinsic(_mm_mfence)
|
||||
#define __TBB_compiler_fence() _ReadWriteBarrier()
|
||||
#define __TBB_full_memory_fence() _mm_mfence()
|
||||
#else
|
||||
#error Unsupported compiler - need to define __TBB_{control,acquire,release}_consistency_helper to support it
|
||||
#endif
|
||||
|
||||
#define __TBB_control_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_acquire_consistency_helper() __TBB_compiler_fence()
|
||||
#define __TBB_release_consistency_helper() __TBB_compiler_fence()
|
||||
|
||||
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
// Workaround for overzealous compiler warnings in /Wp64 mode
|
||||
#pragma warning (push)
|
||||
#pragma warning (disable: 4244 4267)
|
||||
#endif
|
||||
|
||||
extern "C" {
|
||||
__int64 __TBB_EXPORTED_FUNC __TBB_machine_cmpswp8 (volatile void *ptr, __int64 value, __int64 comparand );
|
||||
__int64 __TBB_EXPORTED_FUNC __TBB_machine_fetchadd8 (volatile void *ptr, __int64 addend );
|
||||
__int64 __TBB_EXPORTED_FUNC __TBB_machine_fetchstore8 (volatile void *ptr, __int64 value );
|
||||
void __TBB_EXPORTED_FUNC __TBB_machine_store8 (volatile void *ptr, __int64 value );
|
||||
__int64 __TBB_EXPORTED_FUNC __TBB_machine_load8 (const volatile void *ptr);
|
||||
}
|
||||
|
||||
//TODO: use _InterlockedXXX intrinsics as they available since VC 2005
|
||||
#define __TBB_MACHINE_DEFINE_ATOMICS(S,T,U,A,C) \
|
||||
static inline T __TBB_machine_cmpswp##S ( volatile void * ptr, U value, U comparand ) { \
|
||||
T result; \
|
||||
volatile T *p = (T *)ptr; \
|
||||
__asm \
|
||||
{ \
|
||||
__asm mov edx, p \
|
||||
__asm mov C , value \
|
||||
__asm mov A , comparand \
|
||||
__asm lock cmpxchg [edx], C \
|
||||
__asm mov result, A \
|
||||
} \
|
||||
return result; \
|
||||
} \
|
||||
\
|
||||
static inline T __TBB_machine_fetchadd##S ( volatile void * ptr, U addend ) { \
|
||||
T result; \
|
||||
volatile T *p = (T *)ptr; \
|
||||
__asm \
|
||||
{ \
|
||||
__asm mov edx, p \
|
||||
__asm mov A, addend \
|
||||
__asm lock xadd [edx], A \
|
||||
__asm mov result, A \
|
||||
} \
|
||||
return result; \
|
||||
}\
|
||||
\
|
||||
static inline T __TBB_machine_fetchstore##S ( volatile void * ptr, U value ) { \
|
||||
T result; \
|
||||
volatile T *p = (T *)ptr; \
|
||||
__asm \
|
||||
{ \
|
||||
__asm mov edx, p \
|
||||
__asm mov A, value \
|
||||
__asm lock xchg [edx], A \
|
||||
__asm mov result, A \
|
||||
} \
|
||||
return result; \
|
||||
}
|
||||
|
||||
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(1, __int8, __int8, al, cl)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(2, __int16, __int16, ax, cx)
|
||||
__TBB_MACHINE_DEFINE_ATOMICS(4, ptrdiff_t, ptrdiff_t, eax, ecx)
|
||||
|
||||
#undef __TBB_MACHINE_DEFINE_ATOMICS
|
||||
|
||||
static inline void __TBB_machine_OR( volatile void *operand, __int32 addend ) {
|
||||
__asm
|
||||
{
|
||||
mov eax, addend
|
||||
mov edx, [operand]
|
||||
lock or [edx], eax
|
||||
}
|
||||
}
|
||||
|
||||
static inline void __TBB_machine_AND( volatile void *operand, __int32 addend ) {
|
||||
__asm
|
||||
{
|
||||
mov eax, addend
|
||||
mov edx, [operand]
|
||||
lock and [edx], eax
|
||||
}
|
||||
}
|
||||
|
||||
#define __TBB_AtomicOR(P,V) __TBB_machine_OR(P,V)
|
||||
#define __TBB_AtomicAND(P,V) __TBB_machine_AND(P,V)
|
||||
|
||||
//TODO: Check if it possible and profitable for IA-32 architecture on (Linux and Windows)
|
||||
//to use of 64-bit load/store via floating point registers together with full fence
|
||||
//for sequentially consistent load/store, instead of CAS.
|
||||
#define __TBB_USE_FETCHSTORE_AS_FULL_FENCED_STORE 1
|
||||
#define __TBB_USE_GENERIC_HALF_FENCED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_RELAXED_LOAD_STORE 1
|
||||
#define __TBB_USE_GENERIC_SEQUENTIAL_CONSISTENCY_LOAD_STORE 1
|
||||
|
||||
|
||||
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
|
||||
#pragma warning (pop)
|
||||
#endif // warnings 4244, 4267 are back
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user