[netcf-devel] [PATCH] call aug_load() at most once per second

[Laine Stump]

Previously, netcf would call aug_load() at the start of each public
API call, and rely on augeas quickly determining if the files needed
to be reread based on checking the mtime of all files. With a large
number of files (i.e. several hundred ifcfg files) just checking the
mtime of all files ends up taking quite a long time; enough to turn a
simple "virsh iface-list" of 300 bridges + 300 vlans into a 22 second
ordeal.

With this patch applied, netcf will only call aug_load() at most once
every second, resulting in runtime for virsh iface-list going down to
< 1 second.

The trade-off is that the results of a netcf API call could be up to 1
second out of date (but only due to changes in the config external to
netcf). Since ifcfg files change very infrequently, this is likely
acceptable.
---
src/dutil_linux.c | 8 +++++++-
src/dutil_linux.h | 1 +
2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/dutil_linux.c b/src/dutil_linux.c
index 0850593..24f4d95 100644
--- a/src/dutil_linux.c
+++ b/src/dutil_linux.c
@@ -32,6 +32,7 @@
#include <unistd.h>
#include <ctype.h>
#include <errno.h>
+#include <time.h>

#include <dirent.h>
#include <sys/wait.h>
@@ -151,6 +152,7 @@ int remove_augeas_xfm_table(struct netcf *ncf,
*/
augeas *get_augeas(struct netcf *ncf) {
int r;
+ time_t current_time;

if (ncf->driver->augeas == NULL) {
augeas *aug;
@@ -186,9 +188,12 @@ augeas *get_augeas(struct netcf *ncf) {
}
ncf->driver->copy_augeas_xfm = 0;
ncf->driver->load_augeas = 1;
+ ncf->driver->load_augeas_time = 0;
}

- if (ncf->driver->load_augeas) {
+ current_time = time(NULL);
+ if (ncf->driver->load_augeas &&
+ ncf->driver->load_augeas_time != current_time) {
augeas *aug = ncf->driver->augeas;

r = aug_load(aug);
@@ -207,6 +212,7 @@ augeas *get_augeas(struct netcf *ncf) {
}
ERR_THROW(r > 0, ncf, EOTHER, "errors in loading some config files");
ncf->driver->load_augeas = 0;
+ ncf->driver->load_augeas_time = current_time;
}
return ncf->driver->augeas;
error:
diff --git a/src/dutil_linux.h b/src/dutil_linux.h
index a06a15c..75ac631 100644
--- a/src/dutil_linux.h
+++ b/src/dutil_linux.h
@@ -41,6 +41,7 @@ struct driver {
struct nl_sock *nl_sock;
struct nl_cache *link_cache;
struct nl_cache *addr_cache;
+ time_t load_augeas_time;
unsigned int load_augeas : 1;
unsigned int copy_augeas_xfm : 1;
unsigned int augeas_xfm_num_tables;

[Daniel P. Berrange]

Post by Laine Stump
Previously, netcf would call aug_load() at the start of each public
API call, and rely on augeas quickly determining if the files needed
to be reread based on checking the mtime of all files. With a large
number of files (i.e. several hundred ifcfg files) just checking the
mtime of all files ends up taking quite a long time; enough to turn a
simple "virsh iface-list" of 300 bridges + 300 vlans into a 22 second
ordeal.
With this patch applied, netcf will only call aug_load() at most once
every second, resulting in runtime for virsh iface-list going down to
< 1 second.
The trade-off is that the results of a netcf API call could be up to 1
second out of date (but only due to changes in the config external to
netcf). Since ifcfg files change very infrequently, this is likely
acceptable.
---
src/dutil_linux.c | 8 +++++++-
src/dutil_linux.h | 1 +
2 files changed, 8 insertions(+), 1 deletion(-)

ACK


Regards,
Daniel