How do I let my users apply their own custom formula to a table of data to derive new fields?
如何让我的用户将自己的自定义公式应用于数据表以获取新字段?
I am working on a Django application which is going to store and process a lot of data for subscribed users on the open web. Think 100-10,000 sensor readings in one page request. I am going to be drawing graphs using this data and also showing tables containing it. I expect groups of sensors to be defined by my users, who will have registered themselves on my website (i.e they correspond with a django model).
我正在开发一个Django应用程序,它将在开放网络上为订阅用户存储和处理大量数据。在一个页面请求中考虑100-10,000个传感器读数。我将使用这些数据绘制图形,并显示包含它的表格。我希望我的用户可以定义传感器组,他们将在我的网站上注册(即它们与django模型相对应)。
I would like to allow the user to be able to create fields that are derived from their sensor data (as part of a setup process). For example, the user might know that their average house temperature is (temperature sensor1 + temperature sensor2) / 2 and want to show that on the graph. They might also want something more interesting like solar hot water heated is (temp out - temp in) * flow * conversion constant. I will then save these defined formulas for them and everyone else who views this page of sensor data.
我想允许用户能够创建从其传感器数据派生的字段(作为设置过程的一部分)。例如,用户可能知道他们的平均房屋温度是(温度传感器1 +温度传感器2)/ 2并且想要在图表上显示它。他们可能也想要一些更有趣的东西,如加热的太阳能热水(温度超出温度)*流量*转换常数。然后,我将为他们以及查看此页传感器数据的其他人保存这些已定义的公式。
The main question is how do I define the formula at the centre of the system. Do I just have a user-defined string to define the formula (say 100 chars long) and parse it myself - replace the user defined with an input sample and call it toast?
主要问题是如何在系统的中心定义公式。我是否只有一个用户定义的字符串来定义公式(比如100个字符长)并自己解析它 - 用输入样本替换定义的用户并将其称为toast?
Update
In the end I got just the answer I asked for : A safe way to evaluate a stored user function on the server. Evaluating the same function also on the client when the function is being defined will be a great way to make the UI intuitive.
最后,我得到了我要求的答案:评估服务器上存储的用户功能的安全方法。在定义函数时也在客户端上评估相同的功能将是使UI直观的好方法。
3 个解决方案
#1
Depends on who your clients are.
取决于您的客户是谁。
If this is "open to the public" on the WWW, you have to parse expressions yourself. You can use the Python compiler to compile Python syntax. You can also invent your own compiler for a subset of Python syntax. There are lots of examples; start with the ply project.
如果这是WWW上的“向公众开放”,则必须自己解析表达式。您可以使用Python编译器来编译Python语法。您还可以为Python语法的子集创建自己的编译器。有很多例子;从ply项目开始。
If this is in-house ("behind the firewall") let the post a piece of Python code and exec that code.
如果这是内部的(“防火墙后面”),请发布一段Python代码并执行该代码。
Give them an environment from math import * functionality available.
为他们提供数学导入*功能的环境。
Fold the following around their supplied line of code:
围绕他们提供的代码行折叠以下内容:
def userFunc( col1, col2, col3, ... ):
result1= {{ their code goes here }}
return result1
Then you can exec the function definition and use the defined function without bad things happening.
然后你可以执行函数定义并使用定义的函数而不会发生坏事。
While some folks like to crow that exec is "security problem", it's no more a security problem than user's sharing passwords, and admin's doing intentionally stupid things like deleting important files or turning the power off randomly while your programming is running.
虽然有些人喜欢说那个执行官是“安全问题”,但它不再是用户共享密码的安全问题,而且管理员正在做有意的愚蠢的事情,比如在编程运行时删除重要文件或随机关闭电源。
exec is only a security problem if you allow anyone access to it. For in-house applications, you know the users. Train them.
如果您允许任何人访问它,exec只是一个安全问题。对于内部应用程序,您了解用户。训练他们。
#2
I would work out what operations you want to support [+,-,*,/,(,),etc] and develop client side (javascript) to edit and apply those values to new fields of the data. I don't see the need to do any of this server-side and you will end up with a more responsive and enjoyable user experience as a result.
我会找出你想要支持的操作[+, - ,*,/,(,)等]并开发客户端(javascript)来编辑并将这些值应用到数据的新字段。我认为没有必要在服务器端执行任何操作,因此您最终会获得更快速响应和愉快的用户体验。
If you allow the user to save their formulas and re-load them when they revisit the site, you can get their browser to do all the calculations. Just provide some generic methods to add columns of data which are generated by applying one of their forumla's to your data.
如果您允许用户保存他们的公式并在他们重新访问网站时重新加载它们,您可以让他们的浏览器执行所有计算。只需提供一些通用方法来添加通过将其中一个forumla应用于您的数据而生成的数据列。
I imagine the next step would be to allow them to apply those operations to the newly generated columns.
我想下一步是允许他们将这些操作应用于新生成的列。
Have you considered posting their data into a google spreadsheet? This would save a lot of the development work as they already allow you to define formulas etc. and apply it to the data. I'm not too sure of the data limit (how much data you can post and operate on) mind you.
您是否考虑将其数据发布到Google电子表格中?这将节省大量的开发工作,因为它们已经允许您定义公式等并将其应用于数据。我不太清楚数据限制(你可以发布和操作多少数据)。
#3
Another user asked a similar question in C. In that post, Warren suggested that the formula could be parsed and converted from
另一个用户在C中提出了类似的问题。在那篇文章中,Warren建议可以解析和转换公式
(a + c) / b
进入反向抛光表示法
a c + b /
Which is easier to process.
哪个更容易处理。
In this case, you could intercept the formula model's save and generate the postfix notation from the user-defined formula. Once you have postfix notation, it is fairly straightforward to write a loop that evaluates the formula from left to right.
在这种情况下,您可以截取公式模型的保存并从用户定义的公式生成后缀表示法。一旦你有了后缀表示法,编写一个从左到右评估公式的循环是相当简单的。
As for implementation in Django, the core question remaining is how to map different input fields into the formula. The simple solution would be a model representing the derived field uses a many-to-many relationship with the symbol name ("a", "b" or "c") defined per-input.
至于Django的实现,剩下的核心问题是如何将不同的输入字段映射到公式中。简单的解决方案是表示派生字段的模型使用与每个输入定义的符号名称(“a”,“b”或“c”)的多对多关系。
If performance is really critical, you might somehow further pre-process the postfix formula before applying it to the data.
如果性能非常关键,那么在将后缀公式应用于数据之前,您可能会以某种方式进一步预处理后缀公式。
#1
Depends on who your clients are.
取决于您的客户是谁。
If this is "open to the public" on the WWW, you have to parse expressions yourself. You can use the Python compiler to compile Python syntax. You can also invent your own compiler for a subset of Python syntax. There are lots of examples; start with the ply project.
如果这是WWW上的“向公众开放”,则必须自己解析表达式。您可以使用Python编译器来编译Python语法。您还可以为Python语法的子集创建自己的编译器。有很多例子;从ply项目开始。
If this is in-house ("behind the firewall") let the post a piece of Python code and exec that code.
如果这是内部的(“防火墙后面”),请发布一段Python代码并执行该代码。
Give them an environment from math import * functionality available.
为他们提供数学导入*功能的环境。
Fold the following around their supplied line of code:
围绕他们提供的代码行折叠以下内容:
def userFunc( col1, col2, col3, ... ):
result1= {{ their code goes here }}
return result1
Then you can exec the function definition and use the defined function without bad things happening.
然后你可以执行函数定义并使用定义的函数而不会发生坏事。
While some folks like to crow that exec is "security problem", it's no more a security problem than user's sharing passwords, and admin's doing intentionally stupid things like deleting important files or turning the power off randomly while your programming is running.
虽然有些人喜欢说那个执行官是“安全问题”,但它不再是用户共享密码的安全问题,而且管理员正在做有意的愚蠢的事情,比如在编程运行时删除重要文件或随机关闭电源。
exec is only a security problem if you allow anyone access to it. For in-house applications, you know the users. Train them.
如果您允许任何人访问它,exec只是一个安全问题。对于内部应用程序,您了解用户。训练他们。
#2
I would work out what operations you want to support [+,-,*,/,(,),etc] and develop client side (javascript) to edit and apply those values to new fields of the data. I don't see the need to do any of this server-side and you will end up with a more responsive and enjoyable user experience as a result.
我会找出你想要支持的操作[+, - ,*,/,(,)等]并开发客户端(javascript)来编辑并将这些值应用到数据的新字段。我认为没有必要在服务器端执行任何操作,因此您最终会获得更快速响应和愉快的用户体验。
If you allow the user to save their formulas and re-load them when they revisit the site, you can get their browser to do all the calculations. Just provide some generic methods to add columns of data which are generated by applying one of their forumla's to your data.
如果您允许用户保存他们的公式并在他们重新访问网站时重新加载它们,您可以让他们的浏览器执行所有计算。只需提供一些通用方法来添加通过将其中一个forumla应用于您的数据而生成的数据列。
I imagine the next step would be to allow them to apply those operations to the newly generated columns.
我想下一步是允许他们将这些操作应用于新生成的列。
Have you considered posting their data into a google spreadsheet? This would save a lot of the development work as they already allow you to define formulas etc. and apply it to the data. I'm not too sure of the data limit (how much data you can post and operate on) mind you.
您是否考虑将其数据发布到Google电子表格中?这将节省大量的开发工作,因为它们已经允许您定义公式等并将其应用于数据。我不太清楚数据限制(你可以发布和操作多少数据)。
#3
Another user asked a similar question in C. In that post, Warren suggested that the formula could be parsed and converted from
另一个用户在C中提出了类似的问题。在那篇文章中,Warren建议可以解析和转换公式
(a + c) / b
进入反向抛光表示法
a c + b /
Which is easier to process.
哪个更容易处理。
In this case, you could intercept the formula model's save and generate the postfix notation from the user-defined formula. Once you have postfix notation, it is fairly straightforward to write a loop that evaluates the formula from left to right.
在这种情况下,您可以截取公式模型的保存并从用户定义的公式生成后缀表示法。一旦你有了后缀表示法,编写一个从左到右评估公式的循环是相当简单的。
As for implementation in Django, the core question remaining is how to map different input fields into the formula. The simple solution would be a model representing the derived field uses a many-to-many relationship with the symbol name ("a", "b" or "c") defined per-input.
至于Django的实现,剩下的核心问题是如何将不同的输入字段映射到公式中。简单的解决方案是表示派生字段的模型使用与每个输入定义的符号名称(“a”,“b”或“c”)的多对多关系。
If performance is really critical, you might somehow further pre-process the postfix formula before applying it to the data.
如果性能非常关键,那么在将后缀公式应用于数据之前,您可能会以某种方式进一步预处理后缀公式。