Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion pdi/AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Benoit Martin - CEA (bmartin@cea.fr)
* Add support for const data in `PDI_share`, `PDI_expose` and `PDI_multi_expose`

Jacques Morice - CEA (jacques.morice@cea.fr)
* Add Add operator== in Context::Iterator()
* Add operator== in Context::Iterator()
* Delay data events in multi_expose when the last data is exposed

François-Xavier Mordant - CEA (francois-xavier.mordant@cea.fr)
* Fixed CMake issues, internal API enhancement
Expand Down
2 changes: 2 additions & 0 deletions pdi/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ and this project adheres to
#### Added

#### Changed
* Delay data events in multi_expose when the last data is exposed
[#514](https://github.com/pdidev/pdi/issues/514)

#### Deprecated

Expand Down
1 change: 1 addition & 0 deletions pdi/include/pdi/context.h
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@

#include <pdi/pdi_fwd.h>
#include <pdi/callbacks.h>
//#include <pdi/delayed_data_callbacks.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be remove in the next commit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented-out lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be done in the next commit

#include <pdi/data_descriptor.h>
#include <pdi/datatype_template.h>
#include <pdi/logger.h>
Expand Down
23 changes: 22 additions & 1 deletion pdi/include/pdi/data_descriptor.h
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (C) 2015-2024 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* Copyright (C) 2015-2025 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
Expand Down Expand Up @@ -85,6 +85,14 @@ class PDI_EXPORT Data_descriptor
*/
virtual void share(void* data, bool read, bool write) = 0;

/** Shares some data with PDI
* \param[in,out] data the shared data
* \param read whether read access is granted to other references
* \param write whether write access is granted to other references
* \param delayed_callbacks a class that allow to delay trigger_delayed_data_callbacks for a list of a data
*/
virtual void share(void* data, bool read, bool write, Delayed_data_callbacks& delayed_callbacks) = 0;

/** Shares some data with PDI
* \param[in,out] ref a reference to the shared data
* \param read whether the stored reference should have read access
Expand All @@ -93,6 +101,19 @@ class PDI_EXPORT Data_descriptor
*/
virtual void* share(Ref ref, bool read, bool write) = 0;

/** Shares some data with PDI
* \param[in,out] ref a reference to the shared data
* \param read whether the stored reference should have read access
* \param write whether the stored reference should have write access
* \param delayed_callbacks a class that allow to delay trigger_delayed_data_callbacks for a list of a data
* \return the just shared buffer
*/
virtual void* share(Ref ref, bool read, bool write, Delayed_data_callbacks& delayed_callbacks) = 0;

/** function to call "data_callbacks" for the shared data.
*/
virtual void trigger_delayed_data_callbacks() = 0;

/** Releases ownership of a data shared with PDI. PDI is then responsible to
* free the associated memory whenever necessary.
*/
Expand Down
150 changes: 150 additions & 0 deletions pdi/include/pdi/delayed_data_callbacks.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
/*******************************************************************************
* Copyright (C) 2025 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of CEA nor the names of its contributors may be used to
* endorse or promote products derived from this software without specific
* prior written permission.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
******************************************************************************/

#ifndef PDI_DELAYED_DATA_CALLBACK_H_
#define PDI_DELAYED_DATA_CALLBACK_H_


#include <iostream>
#include <list>
#include <string>

#include <pdi/pdi_fwd.h>
#include "pdi/context.h"
#include "pdi/datatype.h"
#include "pdi/error.h"
#include "pdi/plugin.h"
#include "pdi/ref_any.h"
#include "pdi/scalar_datatype.h"
#include <pdi/context.h>
#include <pdi/data_descriptor.h>
#include <pdi/datatype_template.h>
#include <pdi/ref_any.h>

#include <data_descriptor_impl.h>

#include "global_context.h"

namespace PDI {

class PDI_EXPORT Delayed_data_callbacks
{
/// friend class Global_context;
std::list<std::string> m_datanames;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are iterating over all the elements of this data structure. It probably is better if this is a std::forward_list.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the logger message, I need the size of the list. In std::forward_list, we don't have access to the number of elements.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the size is only used for logger, we can do it differently e.g. nb_elemt = std::distance(m_datanames.begin(), m_datanames.end())


/// The context this descriptor is part of
Global_context& m_context;

/// A descriptor in case of
Data_descriptor_impl** m_descriptor;

public:
/// constructor used in:
/// - desc.share(void* data, bool read, bool write)
/// - desc.share(Ref data_ref, bool read, bool write)
/// to define a "Delayed_data_callbacks" object
Delayed_data_callbacks(Data_descriptor_impl** desc)
: m_descriptor(desc)
, m_datanames(0)
, m_context((*desc)->m_context)
{}

/// constructor used in:
/// - the case of PDI_multi_expose
/// - when a user want to define is own "Delayed_data_callbacks" object
/// Remark: This constructor doesn't work if PDI_init is not called.
Delayed_data_callbacks(Global_context& ctx)
: m_context{ctx}
, m_datanames(0)
, m_descriptor(nullptr)
{}

// In the destructor, we need to throw an error message in case of the callback on the data doesn't work (trigger function)
// (example: error in the config.yml for a plugin, error due to external library incompatibility)
~Delayed_data_callbacks() noexcept(false)
{
try {
this->trigger();
} catch (Error& e) {
if (std::uncaught_exceptions()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that this is useful. I would not use std::uncaught_exceptions() (cf: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4152.pdf)

// An exception is throwing. Print simple message to avoid std::terminate.
m_context.logger().error("Error in the destructor of Delayed_data_callbacks, {}", e.what());
} else {
// No exception is throwing. Throw an error.
throw Error(e.status(), "Error in the destructor of Delayed_data_callbacks, {}", e.what());
}
} catch (const std::exception& e) {
if (std::uncaught_exceptions()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

// An exception is throwing. Print simple message to avoid std::terminate.
m_context.logger().error("Error in the destructor of Delayed_data_callbacks, {}", e.what());
} else {
// No exception is throwing. Throw an error.
m_context.logger().error("Error in the destructor of Delayed_data_callbacks.");
throw;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should not throw in a noexcept method.

}
} catch (...) {
if (std::uncaught_exceptions()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

// An exception is throwing. Print simple message to avoid std::terminate.
m_context.logger().error("Error in the destructor of Delayed_data_callbacks.");
} else {
// No exception is throwing. Throw an error.
m_context.logger().error("Error in the destructor of Delayed_data_callbacks.");
throw;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

}
}
m_descriptor = nullptr;
}

// add element to the list
void add_dataname(const std::string& name) { m_datanames.emplace_back(name); }

// trigger_delayed_data_callbacks()
void trigger()
try {
int i = 0;
size_t number_of_elements = m_datanames.size();
for (; !m_datanames.empty(); m_datanames.pop_front()) {
auto&& element_name = m_datanames.front();
m_context.logger().trace("Trigger data callback `{}' ({}/{})", element_name.c_str(), ++i, number_of_elements);

if (!m_descriptor) {
m_context[element_name.c_str()].trigger_delayed_data_callbacks();
} else {
(*m_descriptor)->trigger_delayed_data_callbacks();
}
}
} catch (Error& e) {
this->cancel(); // clear the m_datanames
throw Error(e.status(), "Error in trigger data callback: `{}'", e.what());
} catch (...) {
this->cancel(); // clear the m_datanames
throw;
}

// clear the element in the list
void cancel() { m_datanames.clear(); }
}; // class Delayed_data_callbacks

} // namespace PDI
#endif // PDI_DELAYED_DATA_CALLBACK_H_
5 changes: 5 additions & 0 deletions pdi/include/pdi/pdi_fwd.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,11 @@ using Datatype_template_sptr = std::shared_ptr<const Datatype_template>;

using Datatype_sptr = std::shared_ptr<const Datatype>;

/**
* A class to delayed the moment to call call_data_callback
*/
class Delayed_data_callbacks;

/** A data descriptors with a name and a value, it contains an implicit type
* template that is used when exposing untyped data
*/
Expand Down
63 changes: 55 additions & 8 deletions pdi/src/data_descriptor_impl.cxx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (C) 2015-2024 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* Copyright (C) 2015-2025 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* Copyright (C) 2021 Institute of Bioorganic Chemistry Polish Academy of Science (PSNC)
* All rights reserved.
*
Expand Down Expand Up @@ -32,6 +32,7 @@

#include "pdi/context.h"
#include "pdi/datatype.h"
#include "pdi/delayed_data_callbacks.h"
#include "pdi/error.h"
#include "pdi/plugin.h"
#include "pdi/ref_any.h"
Expand Down Expand Up @@ -151,13 +152,50 @@ bool Data_descriptor_impl::empty()
return m_refs.empty();
}

void Data_descriptor_impl::trigger_delayed_data_callbacks()
try {
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
if (m_refs.empty() || (m_refs.size() == 1 && metadata()))
throw State_error{"Cannot call trigger_delayed_data_callbacks on a non shared value: : `{}'", m_name};

try {
m_context.callbacks().call_data_callbacks(m_name, ref());
} catch (const exception&) {
m_refs.pop();
throw;
} catch (...) {
m_context.logger().trace("Error in trigger_delayed_data_callbacks\n");
m_refs.pop();
throw;
}

assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
} catch (Error& e) {
throw Error(e.status(), "Unable to execute data_callbacks on data `{}', {}", name(), e.what());
}


void Data_descriptor_impl::share(void* data, bool read, bool write)
try {
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");

Data_descriptor_impl* tmp = this;
Delayed_data_callbacks delayed_callbacks(&tmp);

share(data, read, write, delayed_callbacks);
Comment on lines +182 to +185
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can probably be 1 line and Delayed_data_callbacks may be an rvalue.


assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
} catch (Error& e) {
throw;
}

void Data_descriptor_impl::share(void* data, bool read, bool write, Delayed_data_callbacks& delayed_callbacks)
try {
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
Ref r{data, &free, m_type->evaluate(m_context), read, write};
try {
m_context.logger().trace("Sharing `{}' Ref with rights: R = {}, W = {}", m_name, read, write);
share(r, false, false);
share(r, false, false, delayed_callbacks);
} catch (...) {
// on error, do not free the data as would be done automatically otherwise
r.release();
Expand All @@ -170,6 +208,20 @@ try {
}

void* Data_descriptor_impl::share(Ref data_ref, bool read, bool write)
try {
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
Data_descriptor_impl* tmp = this;
Delayed_data_callbacks delayed_callbacks(&tmp);
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
return share(data_ref, read, write, delayed_callbacks);
Comment on lines +213 to +216
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above. delayed_callbacks can probably an rvalue.

} catch (Error& e) {
throw Error(e.status(), "Unable to share `{}', {}", name(), e.what());
;
} catch (...) {
throw std::runtime_error("void* Data_descriptor_impl::share(Ref data_ref, bool read, bool write)");
}

void* Data_descriptor_impl::share(Ref data_ref, bool read, bool write, Delayed_data_callbacks& delayed_callbacks)
try {
assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
// metadata must provide read access
Expand Down Expand Up @@ -201,12 +253,7 @@ try {
throw Right_error{"Unable to grant requested rights"};
}

try {
m_context.callbacks().call_data_callbacks(m_name, ref());
} catch (const exception&) {
m_refs.pop();
throw;
}
delayed_callbacks.add_dataname(m_name);

assert((!metadata() || !m_refs.empty()) && "metadata descriptors should always keep a placeholder");
return result;
Expand Down
12 changes: 11 additions & 1 deletion pdi/src/data_descriptor_impl.h
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*******************************************************************************
* Copyright (C) 2015-2024 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* Copyright (C) 2015-2025 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
Expand Down Expand Up @@ -32,6 +32,7 @@
#include <paraconf.h>

#include <pdi/pdi_fwd.h>
//#include <pdi/delayed_data_callbacks.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove if not needed

#include <pdi/data_descriptor.h>
#include <pdi/datatype_template.h>
#include <pdi/ref_any.h>
Expand All @@ -43,10 +44,13 @@ namespace PDI {
class PDI_EXPORT Data_descriptor_impl: public Data_descriptor
{
friend class Global_context;
friend class Delayed_data_callback;
friend class Descriptor_test_handler;


struct PDI_NO_EXPORT Ref_holder;

public:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why public ?

/// The context this descriptor is part of
Global_context& m_context;

Expand Down Expand Up @@ -91,8 +95,14 @@ class PDI_EXPORT Data_descriptor_impl: public Data_descriptor

void share(void* data, bool read, bool write) override;

void share(void* data, bool read, bool write, Delayed_data_callbacks& delayed_callbacks) override;

void* share(Ref ref, bool read, bool write) override;

void* share(Ref ref, bool read, bool write, Delayed_data_callbacks& delayed_callbacks) override;

void trigger_delayed_data_callbacks() override;

void release() override;

void* reclaim() override;
Expand Down
2 changes: 2 additions & 0 deletions pdi/src/global_context.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
#include "pdi/callbacks.h"
#include "pdi/context.h"
#include "pdi/context_proxy.h"
//#include "pdi/delayed_data_callbacks.h"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

#include "pdi/data_descriptor.h"
#include "pdi/logger.h"
#include "pdi/plugin.h"
Expand All @@ -49,6 +50,7 @@ namespace PDI {
class PDI_EXPORT Global_context: public Context
{
private:
friend class Delayed_data_callbacks;
friend class Data_descriptor_impl;

/// The singleton Context instance
Expand Down
Loading