Skip to content

Conversation

jan-song
Copy link
Contributor

@jan-song jan-song commented Apr 6, 2020

相关issue #666 #700

@request-info
Copy link

request-info bot commented Apr 6, 2020

你好!感谢你反馈的问题/bug,但是你的描述好像是空的,我们需要你完整的信息,这样才能帮你解决问题 如果不知道怎么写,在新建issue的时候有若干个模板可供选择,祝好!

@jan-song
Copy link
Contributor Author

jan-song commented Apr 6, 2020

1、生成回滚SQL,执行用户需要有grant select any transaction 权限,需要有grant execute on dbms_logmnr 权限
2、数据库需开启最小化附加日志alter database add supplemental log data;
3、需为归档模式;开启附件日志会增加redo日志量,一般不会有多大影响,需评估归档磁盘空间,redo磁盘IO性能
4、sql tuning advisor需要grant advisor权限

@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 7, 2020

问题不大, 考虑下我的这几个修改意见, 不要重开pr, 直接在你的分支上提交推送就行.

@LeoQuote LeoQuote added area/engine enhancement New feature or request and removed 请提供更多信息 labels Apr 7, 2020
@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 7, 2020

😅 单元测试也要改一下, 这个比我上面提的要重要一些, 我看还有一些比较简单的引用未声明变量的问题, 一定要修. https://travis-ci.org/github/hhyo/Archery/builds/672116656 看一下这里, 失败的几个都要改到成功, 你本地也可以跑 python manage.py test -v 3

感谢贡献.

@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 7, 2020

最近 travis-ci 是不是不好使了, 感觉master 上好几次构建都没把状态同步过来, 不行就换 circleci 看看.

@hhyo
Copy link
Owner

hhyo commented Apr 8, 2020

这个pr还需要@ericruan-cn 看下,感觉回滚了他前面的提交

@ericruan-cn
Copy link
Collaborator

ericruan-cn commented Apr 8, 2020 via email

@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 8, 2020

这个...你解决一下吧,他的都merge进master了

@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 9, 2020

最好rebase下 master 然后force push,有一些把之前修改回滚的注意一下。

具体可以看这里的教程 : https://wizardforcel.gitbooks.io/atlassian-git-tut/content/5.1-%E4%BB%A3%E7%A0%81%E5%90%88%E5%B9%B6%EF%BC%9AMerge%E3%80%81Rebase-%E7%9A%84%E9%80%89%E6%8B%A9.html

@hhyo hhyo requested a review from ericruan-cn April 13, 2020 04:08
if close_conn:
self.close()
return execute_result

def backup(self,workflow_id,cursor,begin_time,end_time):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

几个建议和问题:

  1. 参数和上面的用法没对上, 这里是必选参数, 上面的是可选参数.
  2. 这里建议直接传 workflow 对象过去
  3. 这里的cursor 是不是必须传的, 如果是必须传, 建议改一个有辨识度的名字, 和 backup 的cursor 区分开来, 如果不是必须传, 可以用self.get_connection get connection 然后再拿到cursor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为了获取Oracle当前的会话信息cursor是必须传的 ,这个cursor就是执行的cursor,必须是同一个会话,才能抓取到回滚会话的信息,当然也可以关闭执行SQL会话,将需要的会话信息查询出来做参数传过去,这样备份需要新开Oracle的连接会话,多一次fork process开销,代码也更麻烦

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参数的问题太专业这边没看懂你指的是什么,一直做 的Oracle运维,没做过研发,抱歉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的,这个要想获得备份语句,需要和已经执行的上线sql在同一会话里面,要不然不方便从v$logmnr_contents 获取到undo sql

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def backup(self, workflow, current_cursor, begin_time, end_time):

self.backup(workflow, cursor, begin_time, end_time)

# 创建备份库连接
try:
conn = self.get_backup_connection()
cur = conn.cursor()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里改成 backup_cursor = conn.cursor() 后面 curcursor 会看花眼

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

本地部署的代码(1.7.8版本)这边加上了Oracle执行计划查看支持(pr已经实现),Oracle数据修改SQL审核(explain plan for 实现),改动的太多了,我按上面要求重新整理测试后再提交一份吧,这个版本上线的Oracle存储过程和本地1.7.8版本的改动不兼容,后续打算把Oracle的sql tuning advisor也集成到SQL优化功能上去,前端代码暂时没看懂,哪位大神能给点思路?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

那块前端代码比较乱,看你需要展示什么信息,如果能做成通用的格式,可以扩充一下现有的engine,具体可以添加一下我的个人微信

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好吧, 下次你提的时候记得在你的fork里新开一个分支, master 我们没法帮你修改, 一般会有保护.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oracle的sql tuning advisor是Oracle自带的存储过程,执行普通的查询SQL就能获取到SQL优化报告,展示信息其实就是个查询SQL的返回结果。Oracle执行计划获取和这个思路是一样的,执行Oracle特定的存储过程后返回一个clob类型的查询结果。包括Oracle实例性能诊断优化建议报告(ADDM),AWR报告,都是可以通过调用Oracle存储过程的形式获取到,如果要丰富Oracle的功能确实需要做个通用格式,能力有限,完全没思路要怎么弄

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

那实现和返回格式都参考sql优化内的执行计划展示就行

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oracle要有新功能了,好期待。
我准备五一研究一下archery的源代码,希望能看懂。

@LeoQuote LeoQuote changed the base branch from master to oracle April 21, 2020 13:45
@LeoQuote
Copy link
Collaborator

LeoQuote commented Apr 22, 2020

我把这个pr 的base 改成orable 了, 差不多改好后就merge , merge 完了我帮你整理一下再merge 进 master

Oracle执行计划查看支持
oracle 新增SQL tunning advisor
添加Oracle sql tuning advisor
oracle sql tuning advisor支持
添加Oracle SQL类型判断
支持Oracle select、update\delete\insert\create table\create index的语法审核。支持数据修改备份功能。支持执行计划查看。修复查询bug
@jan-song jan-song changed the title 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor Apr 24, 2020
@jan-song jan-song changed the title 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor,Oracle支持支持计划查看 Apr 24, 2020
@jan-song jan-song changed the title 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor,Oracle支持支持计划查看 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor,Oracle支持执行计划查看 Apr 24, 2020
@jan-song
Copy link
Contributor Author

改好了,大神们一起看看有啥问题没,sql tuning advisor哪位帮忙优化一下显示效果,有些缩进丢失了,Oracle客户端sqlplus和其它官方工具连接没这个问题

@jan-song jan-song changed the title 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor,Oracle支持执行计划查看 支持Oracle数据修改备份SQL功能,支持explain,with查询语法,对Oracle表结构和schema显示进行排序,支持sql tuning advisor,Oracle支持执行计划查看,支持Oracle update/insert/delete/create table/create index的语法SQL审核 Apr 24, 2020
@czxin788
Copy link
Contributor

好期待,对oracle要有增强功能了。感谢jan-song、hhyo和LeoQuote的无私奉献。

@hhyo
Copy link
Owner

hhyo commented Apr 25, 2020

diff的结果有点奇怪,能rebase一下吗?

@LeoQuote
Copy link
Collaborator

@hhyo 我改了base,直接merge进来自己rebase吧。

@czxin788
Copy link
Contributor

你好,我给执行sql的用户授予dba权限,是可以在archery里面看到生产的回滚语句;
但是我把dba角色去掉,然后授予如下权限,发现在archery里面不能生产回滚sql;

grant connect,resource to czx;
grant select any transaction to czx;
grant execute on dbms_logmnr to czx;
请问,我还需要授予什么角色

@jan-song
Copy link
Contributor Author

@czxin788
权限没细测试过,暂时没环境测试,可以看看archery日志报错,这边是按如下权限授权的
grant execute on dbms_logmnr to deploy;
grant EXECUTE ANY INDEXTYPE to deploy;
grant DROP ANY PROCEDURE to deploy;
grant DROP ANY VIEW to deploy;
grant DROP ANY INDEX to deploy;
grant INSERT ANY TABLE to deploy;
grant UNDER ANY VIEW to deploy;
grant DROP ANY INDEXTYPE to deploy;
grant ALTER ANY INDEXTYPE to deploy;
grant CREATE MATERIALIZED VIEW to deploy;
grant CREATE SESSION to deploy;
grant DEBUG ANY PROCEDURE to deploy;
grant ALTER ANY PROCEDURE to deploy;
grant CREATE ANY TABLE to deploy;
grant FLASHBACK ANY TABLE to deploy;
grant CREATE ANY INDEXTYPE to deploy;
grant CREATE PROCEDURE to deploy;
grant BACKUP ANY TABLE to deploy;
grant CREATE TABLE to deploy;
grant MERGE ANY VIEW to deploy;
grant CREATE ANY MATERIALIZED VIEW to deploy;
grant ALTER ANY INDEX to deploy;
grant COMMENT ANY TABLE to deploy;
grant MANAGE TABLESPACE to deploy;
grant SELECT ANY DICTIONARY to deploy;
grant UNDER ANY TABLE to deploy;
grant CREATE INDEXTYPE to deploy;
grant DROP ANY MATERIALIZED VIEW to deploy;
grant CREATE ANY VIEW to deploy;
grant CREATE VIEW to deploy;
grant DELETE ANY TABLE to deploy;
grant SELECT ANY TABLE to deploy;
grant DROP ANY TABLE to deploy;
grant ALTER TABLESPACE to deploy;
grant CREATE TABLESPACE to deploy;
grant ALTER ANY MATERIALIZED VIEW to deploy;
grant CREATE ANY PROCEDURE to deploy;
grant CREATE ANY INDEX to deploy;
grant UPDATE ANY TABLE to deploy;
grant LOCK ANY TABLE to deploy;
grant ALTER ANY TABLE to deploy;
grant DROP ANY SEQUENCE to deploy;
grant CREATE SEQUENCE to deploy;
grant ALTER ANY SEQUENCE to deploy;
grant SELECT ANY SEQUENCE to deploy;
grant CREATE ANY SEQUENCE to deploy;
grant select any transaction to deploy;
grant advisor to deploy;

@czxin788
Copy link
Contributor

czxin788 commented Nov 21, 2020

感谢,已经验证,想要archery能看到回滚sql,必须开启如下三个权限,缺一不可:
grant select any transaction to deploy;
grant execute on dbms_logmnr to deploy;
grant SELECT ANY DICTIONARY to deploy;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/engine enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants