This is based on #3414 because it has a convenient helper for populating the table headings. (Should we be using the datastore ids or the labels for the table headings now?)

This PR is really good stuff, i would ask if this could be patched back to 2.5 version. It has better support for displaying numbers for other languages. https://datatables.net/reference/option/language
as we have few client requests to display decimals using 'comma' instead of 'dot'

Great stuff!

Some observations you may want to consider. These comments are based on a dev instance with this PR applied:

• can we make it work with "Add Filter"? The "Add Filter" widget is quite powerful as it pulls the valid range of values from the DB.
• The column name mapping is off by 1 as it doesn't account for the "_id" column
• Make the Responsive extension an optional feature that can be setup during view definition, and have Scrolling the default behavior for recline.js users are used to that interaction.
• Apply FixedColumn and FixedHeader extensions, freezing in place the header row and the "_id" column in place
• Add "Showing 1 to X of N entries (filtered from...." blurb) on header as well. Again, this is more for transition reasons as the existing recline.js data explorer has its nav elements on the header.
• Apply KeyTable extension for keyboard nav within table, again for legacy reasons (recline.js does this)
• Consider adding the Button extension and the Export and Print plugins (https://datatables.net/extensions/buttons/examples/initialisation/export). Coupled with Search filtering, these capabilities will raise data table's utility beyond viewing data!
• [Future Enhancement] datatables.mark.js? :) (https://datatables.net/blog/2017-01-19)

Finished the all but the "future enhancement" one from above. Cherry-picked #3467 over for the speed-up too.

I still need to add the interactive filter embedded in the view like on the recline grid, and enable handling for multiple column sorting. Neither should present any significant problems.

This is feature complete. looking for direction regarding required tests+docs

This is looking fantastic.

For tests, can I suggest this be one of the rare things that gets one of the mocha UI tests? It'd probably be a good way to confirm the basic functionality from top to bottom.

Section on the docs: http://docs.ckan.org/en/latest/maintaining/data-viewer.html

Depends on #3414 and #3467

I need to move JS/CSS into vendor directory. I might also switch to the bootstrap theme to fit in with the rest of the look and feel a little better.

My vote goes to making this a stand-alone extension as well, so that older installations can use it, as well as encourage third-party contributions.

certainly can do. I'll name the stand-alone extension slightly differently to avoid conflicts and copy the helper that's on master into the extension under a different name as well.

@TkTech it's this or JS-based package management. I'm just following http://docs.ckan.org/en/latest/contributing/frontend/index.html#frontend-development-guidelines

For comparison the recline JS is 70k lines or so, but that includes grid, graps and maps

@wardi

Some initial comments, just from playing with it on the frontend:

• The export buttons always appear for me, even when not selecting the option in the form, also they are not properly aligned (perhaps display them at the same level of the row number selector to avoid some white space):
  
• The PDF option hangs for me. How are the exports handled? I don't see any request so I assume it's all done on the browser?
• The whole UI including the pagination under the table should remain in place, and the actual rows viewport should be the one that has scroll bars when overflowing. data-scroll-y='400px' seems to work well for me but you might want to test it a bit (we can probably get rid of data-fixed-header="true" then, including the library requirement).

I guess that the overall styling and format could be improved via CSS later?

100k lines later, all the vendor js and css is now included.

Not sure if it's a big win but perhaps we can remove all *.semanticui.*, *.foundation.*, *.jqueryui.* files as we are not using them?

I'll have a look at the internals and add more comments.

Yes, the export buttons are all "what you see on the page rigtht now" and done in the browser. If you don't want those included (it's probably a big part of the JS load) we could take it out of this PR and people that need it could add it themselves, like other datatables extensions.

If you don't want those included (it's probably a big part of the JS load) we could take it out of this PR and people that need it could add it themselves, like other datatables extensions.

I think this might be a good idea. It is a neat feature but we also need to think about how it relates to the existing download/dump of the database (are the CSVs generated the same?). The UX becomes confusing with too many places to dowload the data from.
I'm not saying it can not be added later but let's keep this PR simpler and iterate.

Other than that and my previous comments about the viewport and I'm happy for it to go.

Also yes, we should start thinking about using a package manager for js requirements and use it as part of the install.

Fixed show/hide columns feature, user-provided filters and removed export buttons

I like how it works, but have one concern about filtering. Is there any simple way to add some delay before changes of search field triggers update of view? Real-time search would be cool with cached results, but hitting server for each letter (if you typing quite slow) is the only disadvantage in comparison with recline data grids.
PS I do not want this to hold PR. this is not critical(for my opinion) - do not think, that search is used so frequently. If such delay cannot be implemented with small changes of config, let's discus and probably change this point in another PR. But if it can - it would be nice to have it

@smotornyuk looks like it defaults to 400ms https://datatables.net/reference/option/searchDelay do you still think that's going to cause a problem? It would be easy to pass in another value

No, i think that this is quite reasonable value.
PS But this is not exactly what i meant. This parameter only restricts search frequency(i.e, if you will type random letters very fast, you'll notice new ajax request to server every 400ms). I'm wandering, whether it possible to make debounce( if you have any experience with reactive programming, there is example )- do not trigger any request unless there are no any activity from user for at least 400ms

@smotornyuk sure, that would be best implemented as a change to the datatables code (or possibly a separate JS extension) because it would be nice for all datatables users. I'll merge this now and that sort of thing can be a future improvement.

@wardi Great extension. I tried today setting it as the default for auto-creation for datastore views, and it almost works. It throws a 500 with this:

StatementError: <ckan.lib.navl.dictization_functions.Missing object at 0x7f408ab2b0d0> is not JSON serializable (original cause: TypeError: <ckan.lib.navl.dictization_functions.Missing object at 0x7f408ab2b0d0> is not JSON serializable) u'INSERT INTO resource_view (id, resource_id, title, description, view_type, "order", config) VALUES (%(id)s, %(resource_id)s, %(title)s, %(description)s, %(view_type)s, %(order)s, %(config)s)' [{'description': u'', 'title': u'Table', 'resource_id': u'3423dd5d-e4e9-4612-b226-a42e7c88d431', 'view_type': u'datatables_view', 'config': {u'filterable': True, u'responsive': False, u'show_fields': <ckan.lib.navl.dictization_functions.Missing object at 0x7f408ab2b0d0>}, 'order': 0}]

Which raises two questions:

1. How do I (in plugin.py) force a default of [] for show_fields (it's in there in the schema)
2. What should the default behaviour actually be? Is there a facility in the default-setting process to do a query on the datastore resource and get the fields? Or would a [] OR "all" option be possible in the schema?

Problem:

'config': {
 u'filterable': True, 
u'responsive': False, 
u'show_fields': <ckan.lib.navl.dictization_functions.Missing object at 0x7f408ab2b0d0>}

Is reclineview going to be removed or is it gonna stay for a while in master?

@klikstermkd no plans to remove it in the short term. Depending on adoption we might consider making Data Tables the default choice

how to add the plugin?

@jitopdeveloper add this to your ini file:

ckan.plugins = ... datatables_view

ckan.views.default_views =  ... datatables_view

@amercader I've tried it but the result is error 500

@jitopdeveloper Which version of CKAN are you using?

@klikstermkd upgraded to 2.7 and this is few of log
Function _resource_preview() in module ckan.controllers.package has been deprecated and will be removed in a later release of ckan. Resource preview is deprecated. Please use the new resource views
[Tue Jul 11 13:55:00.500560 2017] [:error] [pid 32129:tid 140705945786112] 2017-07-11 13:55:00,500 INFO [ckan.lib.base] /dataset/jumlah-penduduk-miskin-menurut-tahun/resource/ebeae7d0-ba3a-44c1-883e-be003a154a93 render time 0.052 seconds

@amercader @klikstermkd problem solved
thanks