eprintid: 10196071 rev_number: 8 eprint_status: archive userid: 699 dir: disk0/10/19/60/71 datestamp: 2024-08-22 10:02:11 lastmod: 2024-08-22 10:02:11 status_changed: 2024-08-22 10:02:11 type: article metadata_visibility: show sword_depositor: 699 creators_name: Clerico, Eugenio creators_name: Guedj, Benjamin title: A note on regularised NTK dynamics with an application to PAC-Bayesian training ispublished: pub divisions: UCL divisions: B04 divisions: F48 note: This version is the version of record. For information on re-use, please refer to the publisher's terms and conditions. abstract: We establish explicit dynamics for neural networks whose training objective has a regularising term that constrains the parameters to remain close to their initial value. This keeps the network in a lazy training regime, where the dynamics can be linearised around the initialisation. The standard neural tangent kernel (NTK) governs the evolution during the training in the infinite-width limit, although the regularisation yields an additional term that appears in the differential equation describing the dynamics. This setting provides an appropriate framework to study the evolution of wide networks trained to optimise generalisation objectives such as PAC-Bayes bounds, and hence contribute to a deeper theoretical understanding of such networks. date: 2024 date_type: published official_url: https://openreview.net/forum?id=2la55BeWwy oa_status: green full_text_type: pub language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2306430 lyricists_name: Guedj, Benjamin lyricists_id: BGUED94 actors_name: Flynn, Bernadette actors_id: BFFLY94 actors_role: owner full_text_status: public publication: Transactions on Machine Learning Research volume: 2024 number: 04 pagerange: 1-20 citation: Clerico, Eugenio; Guedj, Benjamin; (2024) A note on regularised NTK dynamics with an application to PAC-Bayesian training. Transactions on Machine Learning Research , 2024 (04) pp. 1-20. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10196071/1/1972_A_note_on_regularised_NTK.pdf