检测持续时间值列表中的闪烁
Detect flickering in list of durations values
我正在尝试获取来自定期向我发送值的设备的闪烁数据的平均值。
例如，它在 1 分钟的 window 内向我发送了 5 个值，然后下一个值将在一小时内发送，然后在一分钟内再次发送一个值，然后在几个小时后发送下一个值。
就代码而言，假设我有一个 Tuple 列表。我定义了一个阈值，比如 15 分钟。
var flickeringThreshold = 15;
var flickeringList = new List<(DateTime, int)>(8);
// I'd like to regroup all these because the TimeSpan resulting of
// the substraction of two Dates Values ElementAt(index n+1) - ElementAt(index n)
// is under the threshold.
// By regroup I mean weight average the values and take the date
// when the flickering begins, but this not the issue here.
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 5, DateTimeKind.Local), 3));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 10, DateTimeKind.Local), 5));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 15, DateTimeKind.Local), 3));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 20, DateTimeKind.Local), 5));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 25, DateTimeKind.Local), 3));
// This one lives on her own because the difference with the previous value in the list is over the threshold
flickeringList.Add((new DateTime(2022, 3, 25, 11, 4, 0, DateTimeKind.Local), 2));
// This one is alone no flickering
flickeringList.Add((new DateTime(2022, 3, 25, 11, 4, 30, DateTimeKind.Local), 3));
// This one lives on her own because the difference with the previous value in the list is over the threshold
flickeringList.Add((new DateTime(2022, 3, 25, 12, 7, 25, DateTimeKind.Local), 5));
我解决这个问题的第一个方法是使用 for 循环来比较元素。将有一个布尔值来表示闪烁开始和停止...
我的问题是，对于下面的示例，我似乎无法找到一种方法来在循环时不考虑第 6 个值...
伪代码：
for (int i = 0; i < flickeringList.Count; i++)
{
var level = flickeringList[i].Item2;
var nextLevel = i < flickeringList.Count - 1 ? flickeringList[i + 1] : default;
DateTime forDurationStart = flickeringList[i].Item1;
DateTime forDurationEnd = i < flickeringList.Count - 1 ? (DateTime)flickeringList[i + 1].Item1 : default;
if ((forDurationEnd - forDurationStart).TotalMinutes < flickeringThreshold)
{
// Flickering detected
continue;
}
else
{
// We've gone past flickering...
}
}
我该如何解决我的问题？
我找到了一种方法来过滤掉不需要的行（使用 MoreLINQ 中的滞后）但在此过程中丢失了数据以进行加权平均：
var filteredList = flickeringList
.OrderBy(e => e.Item1)
.Lag(1, (e, lag) => new
{
Event = e,
PreviousItem = lag,
})
.Where(x => x.PreviousItem == default || (x.Event.Item1 - x.PreviousItem.Item1).TotalMinutes > flickeringThreshold)
.Select(x => x.Event);
在我看来，您正在处理一个流，这意味着您不知道下一个值何时到来。所以你唯一能做的就是根据你已经拥有的来决定。
假设 q 是等待下一个值可用的 blocking queue。此外，我们假设每个传入项目都有一个 Time 和 Data 值（分别是事件时间和传入项目的值）。
var lastItem = default;
do
{
var nextItem = q.Dequeue();
DateTime forDurationStart = lastItem != default ? lastItem.Time : default;
DateTime forDurationEnd = nextItem.Time;
// keep track of the last incoming item
// this should be moved if you still need to reference lastItem further down
lastItem = nextItem;
// you can extend this if-else to handle the first item differently
// that is, when forDurationStart == default
if (forDurationStart != default && (forDurationEnd - forDurationStart).TotalMinutes < flickeringThreshold)
{
// Flickering detected
continue;
}
else
{
// We've gone past flickering...
}
} while(lastItem.Data != "quit")
这样，我们总是回头检测闪烁。此外，为了采取好的措施，我引入了一个 quit 命令来跳出循环。希望这有帮助。
如果我没理解错的话，当我们得到一个“读数”时，一段闪烁就开始了，并且包括 threshold 期间的所有后续“读数”。如果在threshold到期之前没有后续读数，那么这不是闪烁。
此解决方案建立在您的简单循环之上，尽管我认为 @amir-keibi 解决方案中基于队列的循环在最终分析中可能更好（您可能希望出队超时等同于 threshold 在接收到下一个数据项之前检测闪烁的结束，而不是等待不确定的时间量）。
我使用单个元组来存储有关闪烁周期的足够信息，因此没有必要存储单个项目。
请注意，该问题引用了“加权平均值”，但没有说明权重是如何计算的，所以我假设它是时间加权的。
// Sample data from original question
var flickeringList = new List<(DateTime, int)>
{
(new DateTime(2022, 3, 25, 9, 2, 5, DateTimeKind.Local), 3),
(new DateTime(2022, 3, 25, 9, 2, 10, DateTimeKind.Local), 5),
(new DateTime(2022, 3, 25, 9, 2, 15, DateTimeKind.Local), 3),
(new DateTime(2022, 3, 25, 9, 2, 20, DateTimeKind.Local), 5),
(new DateTime(2022, 3, 25, 9, 2, 25, DateTimeKind.Local), 3),
// This one lives on her own because the difference with the previous value in the list is over the threshold
(new DateTime(2022, 3, 25, 11, 4, 0, DateTimeKind.Local), 2),
// This one is alone no flickering
(new DateTime(2022, 3, 25, 11, 4, 30, DateTimeKind.Local), 3),
// This one lives on her own because the difference with the previous value in the list is over the threshold
(new DateTime(2022, 3, 25, 12, 7, 25, DateTimeKind.Local), 5),
};
// local func to initialize a flicker period from an item.
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue)
InitFlickerPeriod((DateTime dateTime, int value) valueTuple)
{
return (1, 0, 0, valueTuple.dateTime, valueTuple.dateTime, valueTuple.value);
}
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue )
UpdateToFlickerPeriod(
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue)
periodTuple, (DateTime dateTime, int value) valueTuple)
{
var weightOfPriorPointInSeconds = (valueTuple.dateTime - periodTuple.lastItemDate).TotalSeconds;
return (
itemCount: periodTuple.itemCount+1,
weightedAvg: ((periodTuple.weightedAvg*periodTuple.totalWeight)+(weightOfPriorPointInSeconds*periodTuple.lastItemValue)) / (weightOfPriorPointInSeconds+periodTuple.totalWeight),
totalWeight: periodTuple.totalWeight+weightOfPriorPointInSeconds,
firstItemDate: periodTuple.firstItemDate,
lastItemDate: valueTuple.dateTime,
lastItemValue: valueTuple.value
);
}
void DoSomethingWithFlickerPeriod(
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue) flickerTuple)
{
Console.WriteLine($"Flicker consisting {flickerTuple.itemCount} readings with a weightedAvg value of {flickerTuple.weightedAvg} started at {flickerTuple.firstItemDate} and ended at {flickerTuple.lastItemDate}."); }
// Define the threshold
var threshold = TimeSpan.FromMinutes(15);
// In keeping with the question, use a Tuple to store details of a period of flickering. The items represent:
// the number of data items in the weightedAvg;
// the 'weightedAvg';
// the total weight of the weighted average;
// the DateTime of the first item;
// the DateTime of the last item;
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue) flickerPeriod = default;
foreach ((DateTime dateTime, int value) flickerDataPoint in flickeringList)
{
if (flickerPeriod == default)
{
// this must be the first 'flicker' in the period of flickering. Init the filckerPeriod.
flickerPeriod = InitFlickerPeriod(flickerDataPoint);
continue;
}
// If the current data point occurred within the threshold of time since the last one, then we add it to the flicker period.
if ((flickerDataPoint.dateTime - flickerPeriod.firstItemDate) <= threshold)
{
flickerPeriod = UpdateToFlickerPeriod(flickerPeriod, flickerDataPoint);
continue;
}
// If we get here, then the new data point does not belong with the prior period of flickering.
// Assume that flickering occurs only if there is more than one value.
if (flickerPeriod.itemCount > 1)
DoSomethingWithFlickerPeriod(flickerPeriod);
// ...and re-initialize the flickerPeriod.
flickerPeriod = InitFlickerPeriod(flickerDataPoint);
}
// Deal with the final period if there is one.
if (flickerPeriod != default && flickerPeriod.itemCount > 1)
DoSomethingWithFlickerPeriod(flickerPeriod);
生成以下输出（与示例数据中的注释略有不同 - 也许我误解了）：
Flicker consisting 5 readings with a weightedAvg value of 4 started at 25/03/2022 09:02:05 and ended at 25/03/2022 09:02:25.
Flicker consisting 2 readings with a weightedAvg value of 2 started at 25/03/2022 11:04:00 and ended at 25/03/2022 11:04:30.
我正在尝试获取来自定期向我发送值的设备的闪烁数据的平均值。
例如，它在 1 分钟的 window 内向我发送了 5 个值，然后下一个值将在一小时内发送，然后在一分钟内再次发送一个值，然后在几个小时后发送下一个值。
就代码而言，假设我有一个 Tuple
var flickeringThreshold = 15;
var flickeringList = new List<(DateTime, int)>(8);
// I'd like to regroup all these because the TimeSpan resulting of
// the substraction of two Dates Values ElementAt(index n+1) - ElementAt(index n)
// is under the threshold.
// By regroup I mean weight average the values and take the date
// when the flickering begins, but this not the issue here.
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 5, DateTimeKind.Local), 3));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 10, DateTimeKind.Local), 5));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 15, DateTimeKind.Local), 3));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 20, DateTimeKind.Local), 5));
flickeringList.Add((new DateTime(2022, 3, 25, 9, 2, 25, DateTimeKind.Local), 3));
// This one lives on her own because the difference with the previous value in the list is over the threshold
flickeringList.Add((new DateTime(2022, 3, 25, 11, 4, 0, DateTimeKind.Local), 2));
// This one is alone no flickering
flickeringList.Add((new DateTime(2022, 3, 25, 11, 4, 30, DateTimeKind.Local), 3));
// This one lives on her own because the difference with the previous value in the list is over the threshold
flickeringList.Add((new DateTime(2022, 3, 25, 12, 7, 25, DateTimeKind.Local), 5));
我解决这个问题的第一个方法是使用 for 循环来比较元素。将有一个布尔值来表示闪烁开始和停止...
我的问题是，对于下面的示例，我似乎无法找到一种方法来在循环时不考虑第 6 个值...
伪代码：
for (int i = 0; i < flickeringList.Count; i++)
{
var level = flickeringList[i].Item2;
var nextLevel = i < flickeringList.Count - 1 ? flickeringList[i + 1] : default;
DateTime forDurationStart = flickeringList[i].Item1;
DateTime forDurationEnd = i < flickeringList.Count - 1 ? (DateTime)flickeringList[i + 1].Item1 : default;
if ((forDurationEnd - forDurationStart).TotalMinutes < flickeringThreshold)
{
// Flickering detected
continue;
}
else
{
// We've gone past flickering...
}
}
我该如何解决我的问题？
我找到了一种方法来过滤掉不需要的行（使用 MoreLINQ 中的滞后）但在此过程中丢失了数据以进行加权平均：
var filteredList = flickeringList
.OrderBy(e => e.Item1)
.Lag(1, (e, lag) => new
{
Event = e,
PreviousItem = lag,
})
.Where(x => x.PreviousItem == default || (x.Event.Item1 - x.PreviousItem.Item1).TotalMinutes > flickeringThreshold)
.Select(x => x.Event);
在我看来，您正在处理一个流，这意味着您不知道下一个值何时到来。所以你唯一能做的就是根据你已经拥有的来决定。
假设 q 是等待下一个值可用的 blocking queue。此外，我们假设每个传入项目都有一个 Time 和 Data 值（分别是事件时间和传入项目的值）。
var lastItem = default;
do
{
var nextItem = q.Dequeue();
DateTime forDurationStart = lastItem != default ? lastItem.Time : default;
DateTime forDurationEnd = nextItem.Time;
// keep track of the last incoming item
// this should be moved if you still need to reference lastItem further down
lastItem = nextItem;
// you can extend this if-else to handle the first item differently
// that is, when forDurationStart == default
if (forDurationStart != default && (forDurationEnd - forDurationStart).TotalMinutes < flickeringThreshold)
{
// Flickering detected
continue;
}
else
{
// We've gone past flickering...
}
} while(lastItem.Data != "quit")
这样，我们总是回头检测闪烁。此外，为了采取好的措施，我引入了一个 quit 命令来跳出循环。希望这有帮助。
如果我没理解错的话，当我们得到一个“读数”时，一段闪烁就开始了，并且包括 threshold 期间的所有后续“读数”。如果在threshold到期之前没有后续读数，那么这不是闪烁。
此解决方案建立在您的简单循环之上，尽管我认为 @amir-keibi 解决方案中基于队列的循环在最终分析中可能更好（您可能希望出队超时等同于 threshold 在接收到下一个数据项之前检测闪烁的结束，而不是等待不确定的时间量）。
我使用单个元组来存储有关闪烁周期的足够信息，因此没有必要存储单个项目。
请注意，该问题引用了“加权平均值”，但没有说明权重是如何计算的，所以我假设它是时间加权的。
// Sample data from original question
var flickeringList = new List<(DateTime, int)>
{
(new DateTime(2022, 3, 25, 9, 2, 5, DateTimeKind.Local), 3),
(new DateTime(2022, 3, 25, 9, 2, 10, DateTimeKind.Local), 5),
(new DateTime(2022, 3, 25, 9, 2, 15, DateTimeKind.Local), 3),
(new DateTime(2022, 3, 25, 9, 2, 20, DateTimeKind.Local), 5),
(new DateTime(2022, 3, 25, 9, 2, 25, DateTimeKind.Local), 3),
// This one lives on her own because the difference with the previous value in the list is over the threshold
(new DateTime(2022, 3, 25, 11, 4, 0, DateTimeKind.Local), 2),
// This one is alone no flickering
(new DateTime(2022, 3, 25, 11, 4, 30, DateTimeKind.Local), 3),
// This one lives on her own because the difference with the previous value in the list is over the threshold
(new DateTime(2022, 3, 25, 12, 7, 25, DateTimeKind.Local), 5),
};
// local func to initialize a flicker period from an item.
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue)
InitFlickerPeriod((DateTime dateTime, int value) valueTuple)
{
return (1, 0, 0, valueTuple.dateTime, valueTuple.dateTime, valueTuple.value);
}
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue )
UpdateToFlickerPeriod(
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue)
periodTuple, (DateTime dateTime, int value) valueTuple)
{
var weightOfPriorPointInSeconds = (valueTuple.dateTime - periodTuple.lastItemDate).TotalSeconds;
return (
itemCount: periodTuple.itemCount+1,
weightedAvg: ((periodTuple.weightedAvg*periodTuple.totalWeight)+(weightOfPriorPointInSeconds*periodTuple.lastItemValue)) / (weightOfPriorPointInSeconds+periodTuple.totalWeight),
totalWeight: periodTuple.totalWeight+weightOfPriorPointInSeconds,
firstItemDate: periodTuple.firstItemDate,
lastItemDate: valueTuple.dateTime,
lastItemValue: valueTuple.value
);
}
void DoSomethingWithFlickerPeriod(
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue) flickerTuple)
{
Console.WriteLine($"Flicker consisting {flickerTuple.itemCount} readings with a weightedAvg value of {flickerTuple.weightedAvg} started at {flickerTuple.firstItemDate} and ended at {flickerTuple.lastItemDate}."); }
// Define the threshold
var threshold = TimeSpan.FromMinutes(15);
// In keeping with the question, use a Tuple to store details of a period of flickering. The items represent:
// the number of data items in the weightedAvg;
// the 'weightedAvg';
// the total weight of the weighted average;
// the DateTime of the first item;
// the DateTime of the last item;
(int itemCount, double weightedAvg, double totalWeight, DateTime firstItemDate, DateTime lastItemDate, int lastItemValue) flickerPeriod = default;
foreach ((DateTime dateTime, int value) flickerDataPoint in flickeringList)
{
if (flickerPeriod == default)
{
// this must be the first 'flicker' in the period of flickering. Init the filckerPeriod.
flickerPeriod = InitFlickerPeriod(flickerDataPoint);
continue;
}
// If the current data point occurred within the threshold of time since the last one, then we add it to the flicker period.
if ((flickerDataPoint.dateTime - flickerPeriod.firstItemDate) <= threshold)
{
flickerPeriod = UpdateToFlickerPeriod(flickerPeriod, flickerDataPoint);
continue;
}
// If we get here, then the new data point does not belong with the prior period of flickering.
// Assume that flickering occurs only if there is more than one value.
if (flickerPeriod.itemCount > 1)
DoSomethingWithFlickerPeriod(flickerPeriod);
// ...and re-initialize the flickerPeriod.
flickerPeriod = InitFlickerPeriod(flickerDataPoint);
}
// Deal with the final period if there is one.
if (flickerPeriod != default && flickerPeriod.itemCount > 1)
DoSomethingWithFlickerPeriod(flickerPeriod);
生成以下输出（与示例数据中的注释略有不同 - 也许我误解了）：
Flicker consisting 5 readings with a weightedAvg value of 4 started at 25/03/2022 09:02:05 and ended at 25/03/2022 09:02:25. Flicker consisting 2 readings with a weightedAvg value of 2 started at 25/03/2022 11:04:00 and ended at 25/03/2022 11:04:30.