Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Creating an internal data log descriptor.
Quote:I think the first step is getting customizable datalogs working with a couple "hard-coded" packet formats

Yes, lets see what we learn from taking those first couple steps. Even if we roll a descriptor protocol it should not change much, if at all as time goes on and the log data changes etc.

We are almost ready to start making changes.

Finally had a little time last night, we should have something to test and comment on very soon.

Hey Guys,

It's time to post an update on this topic. The good news is that we are just about unchained from the old monolithic data-stream that has dissuaded developers from making changes to the firmware that involved new or removed log-data.

I have successfully tested changing the order of a few variables in the log stream and I am happy to say that EMStune properly interpreted that new data without any flack!

We still need to test other use cases, such as new or removed data(chunk selection). We also need to properly break out the plugin from other EMS's. If anyone would like to have a go check it out here, we would love to hear your feedback or see some patches:

A BIG thanks goes out to Mike and Dave, working-class dudes who still manage to contribute!
(01-31-2014, 06:23 AM)dandruczyk Wrote:
(01-31-2014, 05:59 AM)malcom2073 Wrote: Are you intending to have everything included in the ECU, or just the datalog format?

You could just have the format itself available by request, which would tell you which values were in which order coming back through the datalog packet, or you could have everything available, field name, scalars, offset, size, etc. The latter would be significantly more difficult to generate and parse, but entirely eliminate the need for a datalog definition file on the tuner side.

I think the main concern is with the ability to change the datalog format, so just having the order of fields coming back would be incredibly useful. I'm not against the tuner having to have a generic definition file that lets it know information about what the different fields are, and then the ECU reports what order the fields are coming back in.

I'm partial to the latter approach, i.e. field name, scalars/conversions, size. As the former approach STILL requires the tuning side to have fore-knowledge in order to be able to parse, and that can change with FW over time. I envision a "master table" of all possible dataloggable vars stored in flash, this table contains the following fields (to be adjusted as needed) that is compiled into the firmware (.i.e. generated as a json string as part of the build process?)

1. ID (numeric, 16 bit, must be unique)
2. Name (charactor, 16 chars?)
3. size (uchar 8 bit enumeration) U08, S08, U16, S16, U32, S32, U64, S64
4. multiplier
5. adder
6. Description. (char, freeform up to 255 chars?)
... ?

Fields 4 and 5 are for conversion of "ECU units" to "real world units". In the cases where those are one in the same, multiplier would be 1, adder would be zero. Notice the lack of an "offset". this is because you can request the values returned in any particular order below.

Several default definitions would be defined that return a subset of the master list, with ID's in a reserved block (let's say ID's 1-10 are "fixed" (compiled it?) lists, and id's 11-99 are user settable lists). I propose API calls to return the master list above (with all details), the current datalog table ID (an ID number), followed by a "detailed datalog table request" that returns a list of each of the 6 fields above for the requested table, and the ability to set datalog "tables", by feeding in a LIST of ID's in the order that you want them returned (i.e. setdataLogID<DatalogTableID><numberofvalues><val1><val2><valN><cksum>. Thus you have maximum flexibility, on the tuning side to say. I only want ID 2, 16, 18,21, and 36, assign that to a datalog table ID X i.e. 42, tell the ECU that's the default datalog table ID to be used and on each datalog request it returns that set.

You would need api calls to do the following:
Return the (compiled in master list), so you know what values are which loggable ID
Return the currently set datalog list ID
Return the list of values in that datalog list ID
Set a list for new loggable ID values for a specific datalog list ID
Set the default datalog list ID

In the UI I envision this is a drag and drop table, left has the master list, right has the way you want them, you drag and drop to suite, then save and you're done.

One big advantage of this method is you can get MUCH faster datalog speed as you're only getting sent what you want to see and nothing that you don't. Teh datalog request SHOULD ALWAYS return the datalog table ID as one of its first parameters so the tuning side knows immediately how to parse it., thus allowing interleaving of multiple datalog request types in the datastream .( i.e. default log type will be streamed continually, but it should be possible to request a different stream type as a "one shot" request to update controls/data sources that don't need ultra fast refresh rates.

-- David
(Another) Newbie alert...

You did ask for thoughts, so here goes...

One additional thing that _might_ prove useful is the ability to set the stream sample frequency. i.e. An API entry point to tell it that you want a 'fresh packet' of sampled data every NNNNN milliseconds (even if there was no change)
It might also help to have an API entry point to turn on / off any given data stream (which might also reference the port that the stream is being delivered to). Eg: Turn ON data stream #23 and direct the output to port#03. (Assumes that stream #23 has previously been 'setup' with corresponding API calls and that there is a physical port numbered #03 that the stream output can be sent to.
This would therefore implicitly allow for the creation of an electronic dashboard (updated at say 100mS intervals on one port) to be running concurrently with an entirely DIFFERENT data stream being used as a data logger to be captured at a different sample rate on a totally independent port (whether CAN, Serial or anything else)

I'd also seriously consider a 'reserved list' of the unique IDs that remain static for ANY LibreEMS ECU. (eg: ID#0001 is always RPM, ID#0002 is always RAW MAP value etc etc). The 'reserved' IDs would be limited to the first 1024 (or so) entries. Anything above that is more 'customized'. (eg: ID#0400 thru ID#040F might be for 16 consecutive EGTs which don't even exist in a 'standard' LibreEMS ECU).
And then reserve another 'block' of ID#s at the top end for 'development purposes' (eg: ID#F000 thru ID#FFFF)
I'd propose setting up an IANA type registry 'authority' be put in place to avoid potential 'conflicts' with ID# duplication once they move out of the 'development' phase

Forum Jump:

Users browsing this thread: 1 Guest(s)