Split array column into rows Use OPENJSON feature in sql server (2016+) to parse json array string into rows. Let's first look this: select * from openjson('["small.jpg","mid.jpg","big.jpg"]') select * from openjson('{"A":"small1.jpg","B":"mid1.jpg","C":"big1.jpg"}') Result: key value type 0 small.jpg 1 1 mid.jpg 1 2 big.jpg 1 key value type A small1.jpg 1 B mid1.jpg 1 C big1.jpg 1 Now look at more complehensive example: create table parts (id int, name varchar(255), images varchar(max)) insert into parts (id,name,images) values (1, 'nail','["small.jpg","mid.jpg","big.jpg"]'), (2, 'screw','{"A":"small1.jpg","B":"mid1.jpg","C":"big1.jpg"}') --split array column to rows declare @tmp table(id int, img varchar(4000)) declare @id int,@img varchar(max) declare csr cursor for select id,images from parts open csr fetch next from csr into @id,@img while @@fetch_status=0 begin insert into @tmp (id,img) select @id,value from openjson(@img) fetch next from csr into @id,@img end close csr deallocate csr select * from @tmp Result: Concat rows to a column Using OPENXML feature in sql server to concat rows into a column. Reuse to script above: --concat rows to a column --using above example select id, '_product_image_gallery', stuff((select ','+img from @tmp where id=t.id for xml path('')),1,1,'') from @tmp t group by id Result:

Yesterday, I was notified there were malicious activity from your website, a huge number of email spam messages is originating from it.211cn.ca. As a result the website was suspended as limited access, which only allow registered IP can access the website. When I logon, notification appeared on the panel. And my.siteground home Detail notification In the review panel I saw a list of files that contains malicious code. GEN|eval_base64decode|92ac7c84794de23675efab25846195ea|07/09/20| /home/hostlike/pubweb211/211it/blyuzewtpd.php GEN|eval_base64decode|92ac7c84794de23675efab25846195ea|07/09/20| /home/hostlike/pubweb211/211it/ndxwcecntb.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|11/09/20| /home/hostlike/pubweb211/211it/vckkmhpjam.php GEN|eval_base64decode|92ac7c84794de23675efab25846195ea|07/09/20| /home/hostlike/pubweb211/211it/lllriasktz.php GEN|eval_base64decode|b200c70d10b280b747353869008d8975|10/09/20| /home/hostlike/pubweb211/211it/guzzmmgqcc.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/cmswprkgwt.php GEN|eval_base64decode|6d9dc920c3142026786fac9ef4a85b7f|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/upmnzusqae.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/awiotljzvb.php HEX|strrev_gzinflate_obfusc|69f9670e8743e996e97b271aa8570044|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/rsixodaduj.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/wcbwqrugyx.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/zrjkkylnkr.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/oqjfkoyenl.php HEX|strrev_gzinflate_obfusc|69f9670e8743e996e97b271aa8570044|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/qbhdyxlhku.php HEX|strrev_gzinflate_obfusc|69f9670e8743e996e97b271aa8570044|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/wgmtivragq.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/hiqobbrcem.php HEX|strrev_gzinflate_obfusc|69f9670e8743e996e97b271aa8570044|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/smkjoogdfw.php HEX|strrev_gzinflate_obfusc|69f9670e8743e996e97b271aa8570044|07/09/20| /home/hostlike/pubweb211/211it/wp-content/plugins/wp-file-manager/lib/files/mmnudpdxaj.php STR|wp_vcd_malware|8125436f73ec321255941993e83d4215|17/05/19| /home/hostlike/pubweb211/211it/wp-content/themes/apostrophe-2-wpcom/functions.php STR|wp_vcd_malware|3bd15df2ee78583e0e5249c20b004570|17/05/19| /home/hostlike/pubweb211/211it/wp-content/themes/twentynineteen/functions.php STR|wp_vcd_malware|89e770805bc10deaaed0bc2b5e5e2044|17/05/19| /home/hostlike/pubweb211/211it/wp-content/themes/apostrophe-2-wpcom-child/functions.php STR|wp_vcd_malware|3342254765f2895b7287d55206c4715a|30/12/19| /home/hostlike/pubweb211/211it/wp-content/themes/twentytwenty/functions.php STR|wp_vcd_malware|6a87f54f37ac47b27764d760559444fe|17/05/19| /home/hostlike/pubweb211/211it/wp-content/themes/twentyseventeen/functions.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|10/09/20| /home/hostlike/pubweb211/211it/hllyqgfyoo.php GEN|eval_base64decode|d2015c782da6ab8e637d8e6fe7318d06|11/09/20| /home/hostlike/pubweb211/211it/F0xAutoConfig/inthagame.php GEN|eval_base64decode|d2015c782da6ab8e637d8e6fe7318d06|22/02/18| /home/hostlike/pubweb211/211it/api/intrigantes.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|10/09/20| /home/hostlike/pubweb211/211it/jsilrrlpdj.php STR|wp_vcd_malware|6fc123ea0b9564867df8f3391206f05c|02/01/20| /home/hostlike/pubweb211/211it/wp-includes/post.php STR|wp_vcd_malware|6e618a387332c93c2f78e08f21c724d3|17/01/20| /home/hostlike/pubweb211/211it/wp-includes/wp-vcd.php HEX|js_url_slider|92ea63f479cf2a449af94368f951010f|06/09/20| /home/hostlike/pubweb211/211it/wp-includes/wp-tmp.php GEN|eval_base64decode|d2015c782da6ab8e637d8e6fe7318d06|17/05/19| /home/hostlike/pubweb211/211it/wp-snapshots/intro.php HEX|eval_gzinflate_b64d_0|8266b96d0ba86770c9f7edf62198a003|07/05/19| /home/hostlike/pubweb211/211it/pages/tk_desk.php HEX|eval_gzinflate_b64d_0|8266b96d0ba86770c9f7edf62198a003|22/02/18| /home/hostlike/pubweb211/211it/tools/tjsend.php GEN|eval_base64decode|d6a7b5db62bf7815a10a17cdf7ddbd4b|10/09/20| /home/hostlike/pubweb211/211it/dvflreeurs.php HEX|remote_content_includer_6|3480dcf23fdfa81ed608897c2680bb4a|07/09/20| /home/hostlike/pubweb211/211it/vynugoevjh.php WP-VCD malware is used to add an admin user, it hides in legitimate WordPress files. The initial version of this threat was uploaded via wp include file called wp-vcd.php. The malicious code is injected into Wordpress core files like functions.php, class.wp.php, post.php, and also injected into WordPress themes, including child theme. It was reported that WP-VCD malware is a malicious code that comes bundled with illegitimate versions of a paid theme/plugin. These pirated themes and plugins incorporate malicious scripts that get inserted once you install them. Once leaked to your website though a nulled theme, it starts spreading to infect other themes on your site and even proliferates to infect every unprotected site hosted on the shared server. If you’ll look for “Free [pugin…

Add some useful extension methods for string manipulations in C#. Make coding more elegent, interesting and productive. string.ToBoolean(); string.ToInt(); string.RemoveSufix(string suffix); string.RemovePrefix(string prefix); string.Remove(string toBeRemoved); string.RemoveSpace(); string.RemoveWhiteSpace();   /// <summary> /// Accept string value: "true", "t", "1" as true, all others will be evaluated as false. /// </summary> /// <returns></returns> public static bool ToBoolean(this string value) { if (string.IsNullOrEmpty(value)) return false; switch (value.Trim().ToLower()) { case "true": return true; case "t": return true; case "1": return true; case "0": return false; case "false": return false; case "f": return false; case "": return false; default: return false; //throw new InvalidCastException("You can't cast that value to a bool!"); } } /// <summary> /// Convert a string to int with default value /// </summary> /// <returns></returns> public static int ToInt(this string value) { if (string.IsNullOrEmpty(value)) return 0; int.TryParse(value, out int i); return i; } /// <summary> /// Remove string suffix /// </summary> /// <param name="suffix"></param> /// <returns></returns> public static string RemoveSufix(this string value, string suffix) { if (value.EndsWith(suffix)) { value.Substring(0, value.Length - suffix.Length); } return value; } /// <summary> /// Remove string prefix /// </summary> /// <param name="prefix"></param> /// <returns></returns> public static string RemovePrefix(this string value, string prefix) { if (value.StartsWith(prefix)) { value.Substring(prefix.Length); } return value; } /// <summary> /// Remove all spaces from a string /// </summary> /// <returns></returns> public static string RemoveSpace(this string value) { return value.Replace(" ", string.Empty); } /// <summary> /// Remove all white spaces from a string, including " ", \t, \r, \n /// </summary> /// <returns></returns> public static string RemoveWhiteSpace(this string value) { return value.Replace(" ", string.Empty) .Replace("\t",…

Zoom meeting recorded video and saved in mp4 format file. But it is not allowed to download. In order to overcome "access denied" error, we have to add hread informations: user agent, cookie, and referer. Here is the source code to download the video using python. import requests url="https://ssrweb.zoom.us/cmr/replay/2020/09/21/2189503564/CD309625-D877-4B35-B99F-34CA4529F0CD/GMT20200920-233653_Tom-Ren-s-_1920x900.mp4?response-content-type=video%2Fmp4&response-cache-control=max-age%3D0%2Cs-maxage%3D86400&data=b87bffbf0d63fc51fb5b53caa6a6e3be04d1fc8b436a138418c4cfce8ff64b49&s001=yes&cid=aw1&fid=eT5a1fFKL6E57JgRK8F681dHTUjCtkgCRQSIhogfoc991lO-o1SPZgBwVzAnFJVok-qWSQ3oTrbpuF_L.01AJKlG7IPx983bD&s002=ly1zSoH_nrGCD_5BpiDHTfumcztfOw0ymT3bgpFePYuEvjANCyhpAsfNHw.bIBlHettrxUuB0Ga&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHBzOi8vc3Nyd2ViLnpvb20udXMvY21yL3JlcGxheS8yMDIwLzA5LzIxLzIxODk1MDM1NjQvQ0QzMDk2MjUtRDg3Ny00QjM1LUI5OUYtMzRDQTQ1MjlGMENEL0dNVDIwMjAwOTIwLTIzMzY1M19Ub20tUmVuLXMtXzE5MjB4OTAwLm1wND9yZXNwb25zZS1jb250ZW50LXR5cGU9dmlkZW8lMkZtcDQmcmVzcG9uc2UtY2FjaGUtY29udHJvbD1tYXgtYWdlJTNEMCUyQ3MtbWF4YWdlJTNEODY0MDAmZGF0YT1iODdiZmZiZjBkNjNmYzUxZmI1YjUzY2FhNmE2ZTNiZTA0ZDFmYzhiNDM2YTEzODQxOGM0Y2ZjZThmZjY0YjQ5JnMwMDE9eWVzJmNpZD1hdzEmZmlkPWVUNWExZkZLTDZFNTdKZ1JLOEY2ODFkSFRVakN0a2dDUlFTSWhvZ2ZvYzk5MWxPLW8xU1BaZ0J3VnpBbkZKVm9rLXFXU1Ezb1RyYnB1Rl9MLjAxQUpLbEc3SVB4OTgzYkQmczAwMj1seTF6U29IX25yR0NEXzVCcGlESFRmdW1jenRmT3cweW1UM2JncEZlUFl1RXZqQU5DeWhwQXNmTkh3LmJJQmxIZXR0cnhVdUIwR2EiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2MDA3NTA5MzB9fX1dfQ__&Signature=J-GEzf4ZyLk~Jlt~05nwpyWnVlw1~KTHasbnBpYaUV5H36d7Vijx~arr0VHNjUvG6jMCqQvjZBht5OnGmKLNKwLCC8sgJsY~dK~~LHyWXtOeSmKCN5OeIftk6YITItvnpyWYujuggXECzcGqis1K~gxa9krlax0J5~OosoKZat33JZISttY3RPAbfO~GeRqfJkDwiTVS1sKIyFdFbc~m-haPN4SlDx1ZDKpDWowYtRIkd4u692cclyp7fgxeIRy9OKyKt3Wfkw-39fTq5TcDJ5u1Yq~o4u0qbeLzASSFDgw7DRpZVnMqODupnhXp1QrZ8886s~ij7GU2oY7rXKzRHA__&Key-Pair-Id=APKAJFHNSLHYCGFYQGIA" content = requests.get(url, headers={ "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36", "Cookie": "_zm_ssid=aw1_c_rWPCvlX3Tcmc8iikrsu0Bg; _zm_mtk_guid=b2c4961722954e1890f35c83dc55fc01; zm_oauth_cluster=aw1; zm_general_guid=""5Rog1fA9Dxwqucr9rINhlQzwlU/sFl0FdC6GZtoSZp4=""; _zm_lang=en-US; zm_gnl_guid=ab1cGxWMcSkw4h2H6yO/DOPyREzINR14Ppkyl7FBOlG1eU5nI2Qnwdltx4ESb0I1odSd; zm_gnl_ruid=udw1HTAmShCudkTwo4ToCA; _zm_csp_script_nonce=JG_kdyI9T-CxO2Y4shUwHA; _zm_launcher=1597953850725; zm_gnl_guid_v2=dffcVktj7guFTpRscXSNCeINKl8xSIlozAf16PavwVWL29lhYi78Ym1sSg3l9Js/VWHPWMBeaBr7Ap6D0q0Z00gLsZb2SrCjP5C8Y=; _zm_page_auth=aw1_c_wCHnZvcWSvuMw-h46T8FIg; _zm_ctaid=51pkTNQpTz-uUZmjm4Rulw.1600739087485.c798cb33478fb73a92bac81a5f6662e2; _zm_chtaid=811; _zm_currency=CAD", "referer": "https://zoom.us/rec/play/P4ZLJUcpCuFLJBpjP7wRiBRHquR8PNIK91C3FRHSPGJ0XX39e6e88ZYmzRLhXq9djw90IHTTTKv0cxfI.QAryZuRQmCV5viBv?continueMode=true&_x_zm_rtaid=51pkTNQpTz-uUZmjm4Rulw.1600739087485.c798cb33478fb73a92bac81a5f6662e2&_x_zm_rhtaid=811" }).content # print(content) counter = 0 with open('./video/第二讲 加拿大房屋保险.mp4', 'wb') as f: # f.write(content) # f.close() for buffer in content.iter_content(chunk_size=4096): f.write(buffer) counter += len(buffer) print(counter)

Unicode expressed like "\u4a44". Chinese words located from 0x3400 to 0x9fa5, including simplified and traditional words. Online tool: Unicode to Chinese covnerter C# Stores unicode data as a string, for each unicode in a string lookup the string by hex or integer number to get the right characters to replace them in the string. string ZHTable = "㐀㐁㐂㐃㐄㐅㐆㐇㐈㐉㐊㐋㐌㐍㐎㐏㐐... ...";   private string GetChar(string unicode) { unicode = unicode.Replace("\\u", ""); var code = Convert.ToInt32(unicode, 16); return ZHTable[code - 13312].ToString(); } var matches = Regex.Matches(text, @"\\u[a-f0-9]{4}"); string result = text; if (matches.Count > 0) { foreach (var m in matches) { var key = m.ToString(); result = result.Replace(key, GetChar(m.ToString())); } }   public string ToChinese(string text) { var matches = Regex.Matches(text, @"\\u[a-f0-9]{4}"); string result = text; if (matches.Count > 0) { foreach (var m in matches) { var key = m.ToString(); result = result.Replace(m.ToString(), GetChar(key)); } } return result; } PHP Stores unicode data in a dictionary like array, for each unicode in a string look up the dictionary to get the right characters to replace them in the string. $unicodedata = [ ... ... 0x3437 => '㐷', 0x3438 => '㐸', 0x3439 => '㐹', 0x343a => '㐺', 0x343b => '㐻', 0x343c => '㐼', ... ... ]; for number calculate as power of 16; for a-f, convert to ascii code - 87, so as a=10, f=15, then calculate as power of 16; for hex letter only has a-f, other than that throw exception. if($this->is_number($u[$i])){ $val += $u[$i] * pow(16, 3-$i); } elseif($this->is_hexletter($u[$i])){ $val += ord($u[$i])-87 * pow(16,3-$i); } to match all unicode, which…

install Flask pip install flask Get Template Writing code Idea: Based on the proper visualization template, feed data from python program to the template, then render the result in the template. 1. Passing data from python to template through javascript. In Python file: from flask import Flask, render_template app = Flask(__name__) @app.route('/') def root(): divdata = '草莓,乌梅,蓝莓,红莓,白梅,话梅,绿莓'; areadata = '555,666,777'; return render_template('bigdata.html', divdata=divdata, areadata=areadata, ... ) if __name__ == '__main__': app.run(debug=True) 2. In template html file Render data as an html element, using js to convert it into js varaibles. <div id='divdata' style="display:none">{{ divdata }}</div> <div id='areadata' style="display:none">{{ areadata }}</div> <script> var divdata = document.getElementById('divdata').innerText; var datadiv = divdata.split(','); var areadata = document.getElementById('areadata').innerText; var dataarea = areadata.split(','); </script> 3. In chart js configuration file: //initialize echarts, and configure the chart var myChart = echarts.init(document.getElementById('echart1')); option = { ... xAxis: [{ type: 'category', data: datadiv, axisLine: { var data = [ { name: '海门', value: dataarea[0]}, { name: '鄂尔多斯', value: dataarea[1]}, { name: '招远', value: dataarea[2]}, { name: '舟山', value: dataarea[3]}, Result

Error handling in sql server was a pain before SQL Server 2005. Starting from  Sql Server 2005 try ... catch blocks was introduced. It was not only giving us a beauty of coding, but greatly improved our development performance. It comes with the following build-in functions: ERROR_NUMBER() ERROR_SEVERITY() ERROR_STATE() ERROR_PROCEDURE() ERROR_LINE() ERROR_MESSAGE() When these functions is called in a CATCH block, they will returns the error information. After working on many projects, I found a best pattern for error handling in Sql Server as the following steps. 1. Create a log table to store error information catch by try...catch blocks, CREATE table sys_log( id int identity primary key, procName varchar(255), essionID int, errorNumber int, severity smallint, errorState smallint, errorLine int, description nvarchar(255), info nvarchar(max), mslangid smallint default 1033, --1033 English, 2052 Chinese timeStemp datetime default getdate()) GO 2. create a stored procedure to catch and log error information: CREATE proc sys_LogError @procname sysname, @info nvarchar(max)='' AS INSERT INTO sys_log(ProcName,SessionID,ErrorNumber,Severity,ErrorState,ErrorLine,Description,info) values ( @procname, @@spid, ERROR_NUMBER(), ERROR_SEVERITY(), ERROR_STATE(), ERROR_LINE(), ERROR_MESSAGE(), @info ) 3. in all stored procedures that need to handle exepctions in transaction using this template: CREATE proc proc_name AS BEGIN tran BEGIN try -- do something print 1/0 commit tran END try BEGIN catch rollback tran declare @info nvarchar(max)='@table='+@table+', @urls='+@urls declare @procname sysname = OBJECT_SCHEMA_NAME(@@PROCID) + '.' + OBJECT_NAME(@@PROCID) exec sys_LogError @procname,@info ---- re-throw exeception if you need --declare @msg nvarchar(2048) = error_message() --declare @severity smallint = error_severity() --declare @state smallint = error_state() --raiserror (@msg, @severity, @state) END catch ERROR_MESSAGE() can be referenced with MASTER.dbo.sysmessages table, mslangid is for identifying…

Based OO principles the private fields should not be accessd by anything outside of the object. However, for some reason we have to access the private fields, a good example is unit testing. With dotnet reflection we could access any fields, members in an object. The accessibility is controled by BindingFlags, specify proper bindingFlags we can access all private fields in a class. To access the private fields in base class we need to add one more level of the type with BaseType. Here is the extension method to get the value of a public and private field in an instance. public static T GetFieldValue<T>(this object obj, string name) { var bindingFlags = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance; var field = obj.GetType().GetField(name, bindingFlags); if (field == null) field = obj.GetType().BaseType.GetField(name, bindingFlags); //father else if (field == null) field = obj.GetType().BaseType.BaseType.GetField(name, bindingFlags); //granpa else if (field == null) field = obj.GetType().BaseType.BaseType.BaseType.GetField(name, bindingFlags); //father of granpa return (T)field?.GetValue(obj); }

Post_name in wordpress is urlencoded, and not showing Chinese chars. urlencode DELIMITER ; DROP FUNCTION IF EXISTS urlencode; DELIMITER | CREATE FUNCTION URLENCODE(str VARCHAR(4096) CHARSET utf8) RETURNS VARCHAR(4096) CHARSET utf8 DETERMINISTIC CONTAINS SQL BEGIN -- the individual character we are converting in our loop -- NOTE: must be VARCHAR even though it won't vary in length -- CHAR(1), when used with SUBSTRING, made spaces '' instead of ' ' DECLARE sub VARCHAR(1) CHARSET utf8; -- the ordinal value of the character (i.e. ñ becomes 50097) DECLARE val BIGINT DEFAULT 0; -- the substring index we use in our loop (one-based) DECLARE ind INT DEFAULT 1; -- the integer value of the individual octet of a character being encoded -- (which is potentially multi-byte and must be encoded one byte at a time) DECLARE OCT INT DEFAULT 0; -- the encoded return string that we build up during execution DECLARE ret VARCHAR(4096) DEFAULT ''; -- our loop index for looping through each octet while encoding DECLARE octind INT DEFAULT 0; IF ISNULL(str) THEN RETURN NULL; ELSE SET ret = ''; -- loop through the input string one character at a time - regardless -- of how many bytes a character consists of WHILE ind <= CHAR_LENGTH(str) DO SET sub = MID(str, ind, 1); SET val = ORD(sub); -- these values are ones that should not be converted -- see http://tools.ietf.org/html/rfc3986 IF NOT (val BETWEEN 48 AND 57 OR -- 48-57 = 0-9 val BETWEEN 65 AND 90 OR -- 65-90 = A-Z val BETWEEN 97 AND 122 OR -- 97-122 =…

Summary The data is stored in javascript variable in 51job website. The crawling could not get right data directly from html, but javascript. Using json plugin to achive the goal. Website: https://search.51job.com/ Search result Html source The data is not in the page html, but javascript code in json format. Xpath Using xpath to look for data on the page is fine, but in the code it returns nothing, because the data is in javascript variable. contentx = le.HTML(content) rets = contentx.xpath('//div[@class="e"]//p[@class="t"]') # returns nothing Crawling with python Using regular expression to parse out the jason data in javascript. ret = re.findall('window.__SEARCH_RESULT__ = (.*?)</script>', content_str)[0] Use json plugin to load ret into json object. d = json.loads(ret) Finally, get the data from json array. Complete code import requests import lxml.etree as le import re import json content = requests.get( url='https://search.51job.com/list/010000,000000,0000,00,9,99,python,2,1.html?lang=c&postchannel=0000&workyear=99&cotype=99&degreefrom=99&jobterm=99&companysize=99&ord_field=0&dibiaoid=0&line=&welfare=', ).content content_str = content.decode('gbk', 'ignore') # <meta http-equiv="Content-Type" content="text/html; charset=gbk"> ret = re.findall('window.__SEARCH_RESULT__ = (.*?)</script>', content_str)[0] d = json.loads(ret) # <class 'dict'> {'top_ads': [], 'auction_ads': [], 'market_ads': [], 'engine_search_result': [{'type': 'engine_search_result', results = d['engine_search_result'] for result in results: print(result)